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ABSTRACT 

This document consists of an interim report and a 
final report which describe the second and third phases of a project 
designed to determine the utility and utilization of sophisticated 
hardware and software capabilities knov:n as advanced instructional 
features (AIFs). Used with an aircrew training device (ATD), AIFs 
permit a simulator instructor to control, monitor, and fabricate 
simulator training missions. Phase II of the project asked 273 
instructor pilots, flight engineers, and radar/navigators from Air 
Training Command (ATC) , Military Airlift Command (MAC), and Strategic 
Air Command (SAC) to rate each of 16 AIFs on five seven-point rating 
scales. Phase III extended the survey to 155 electronic warfare and 
aerial gunnery instructors from ATC, SAC, and TAC training 
facilities. Based on utility and utilization ratings, the T-5 and T-4 
trainers were the most favorably rated by the respondents. They were 
followed, in order, by the F-4G simulator, B-52 weapon system 
trainer, and A-10 simulator. The level of AIF use was affected 
somewhat by hardware and software unreliability, implementation time, 
functional limitations, and design deficiencies; however, the 
perceived value of a feature was the most important determiner of its 
use. It is recommended that: (1) a formal intensive training program 
be established to teach simulator instructors how to use AIFs more 
effectively, and (2) future procurement of AIFs be preceded by a 
detailed front-end analysis that clearly relates AIF capability to 
training needs. Appendices contain the instructional features, 
questionnaires, and five Jata tables. (JB) 
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SUMMARY 



Aircrew training devices (ATDs) are often equipped with sophisticated 
hardware and software capabilities that permit a simulator Instructor (SI) 
to control, monitor, record, and fabricate flight simulation training 
missions. These advanced Instructional features (AIFs) reflect the primary 
role of the ATD as a flight trainer. The training value of an ATO Is a 
function of the degree to which It simulates a particular aircraft and the 
way 1n which It Is used as an Instructional device. AIFs are costly to 
Implement. In order to justify these costs, the fol lowing questions must 
be answered: How frequently are AIFs used? How easy are they to use? Are 
SI'S adequately trained to use AIFs? Oo AIFs have significant training 
value? 

This report describes the second phase of a three-phase project 
designed to obtain answers to these questions by surveying Sis from the Air 
Force Major Connands (MAJCOMs). An on-site survey was administered to 273 
Sis assigned to replacement training units (RTUs) and continuation training 
units (CTUs) at principal Air Training Command (ATC) (T-37, T-38). Military 
Airlift Connand (MAC) (C-5A. C-141, C-130, CH-3, HH-53), and Strategic Air 
Command (SAC) (FB-lllA) ATO sites. The survey requested background 
information along with five seven-point rating scales for evaluating each 
of 16 AIFs. Written comments concerning the 16 AIFs or the ATO were 
solicited. The most striking difference between the Phase I (TAC survey) 
and Phase II results was in the overall magnitude of the ratings. In 
comparison to the TAC Sis, the ATC, MAC, and SAC Sis used AIFs more often, 
found them easier to use, received more training in their use, and 
considered AIFs to be more Important for training. The results suggested 
that TAC's SI training program 1$ less extensive and less structured than 
those of the other MAJCOMs. 

Features such as freeze, reset, motion, environmental, and crash/kill 
override were consistently rated high in utility and utilization, whereas 
features such as automated malfunction insertion, demonstration, 
record/playback, and hard copy were generally rated lower. The level of 
AIF use was affected somewhat by hardware and/or software unreliability, 
implementation time, functional limitations, and design deficiencies. The 
perceived training value of a feature was the most Important determiner of 
its use. 



ERIC 



PREFACE 

This research was conducted to satisfy requirements of Air Force 
Human Resources Laboratory Technical Planning Objective 3. the thrust of 
which Is aircrew training effectiveness. The general objective of this 
thrust 1s to Identify and demonstrate cost-effective simulator training 
strategies and systems to develop and maintain the combat effectiveness of 
Air Force aircrew members. More specifically, the research was conducted 
under the Air Combat Training Research subthrust, the goal of which Is to 
provide a technology base for training high level and quickly perishable 
skills In simulated combat environments. Work Unit 1123-02-34, Development 
and Evaluation of Advanced Instructional Features, addressed a portion of 
this subthrust. Capt L. Welkhorst was the project monUor and Dr. Donald 
J. Polzella, under contract to the University of Dayton Research Institute, 
was the principal investigator. 
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Aeronautical Sy^stems Division (AFSC/ASD), Wright-Patterson AFB, Ohio; 
Headquarters Air Training Command, Randolph AFB, Texas; Headquarters 
Military Airlift Command, Scott AFB, Illinois; and Headquarters Strategic 
Air Command, Offutt AFB, Nebraska. The author gratefully acknowledges the 
assistance of the following individuals: 

Mr. H. Craig McLean (ASD/YWE) 

Maj Buddy Simpson (ATC/XPTTP) 

Capt mke Nickel 1 (3305 SCHS/IDG) 

Maj Joe Burch (MAC/DOT) 

Maj Ron Dukos (HQ MAC/DOTF) 

Lt Col Roy Emerson (SAC/4235 STS) 

Maj Baxter (443 MAW/DOT) 

Russ Cheney (436 AMS/MAAD) 

Dick Parsons (438 AMS/MAAD) 
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Maj Paul Scoville (34 TATG) 
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AIRCREW TRAINING DEVICES: UTILITY AND UTILIZATION 
OF ADVANCED INSTRUCTIONAL FEATURES 

(PHASE II - AIR TRAINING COMMAND. MILITARY AIRLIFT 
COMMAND. AND STRATEGIC AIR COMMAND) 



I. INTRODUCTION 



An aircrew training device (ATD) serves two functions. First, It Is a 
ground-based substitute aircraft that permits student flight crews to flv 
In a safe and carefully controlled environment. More Importantly, an ATD 
Is, as Its name Implies, a teaching machine that Is designed to facilitate 
the acquisition of flight crew skills. In order to fulfill this second 
function, an ATD Is equipped with sophisticated hardware and software 
capabilities that permit a simulator Instructor (SI) to control, monitor, 
and In some cases fabricate simulator training missions. These 
capabilities, which are listed in Table 1, are known as advanced 
Instructional features (AIFs). The list was compiled from several sources, 
but It was drawn primarily from Semple, Cotton, and Sullivan's (1981) 
extensive report describing the AIF capabilities of various military and 
commercial devices. 



Table 1. Advanced Instructional Features 



BRIEFING FEATURES 

Recorded Briefing permits JI to provide a student with Information 
about the simulator and/or a structured training mission through 

audiovisual media presentation.^ 

Demonstration permits SI to demonstrate optimal flying performance by 
means of prerecorded standardized segments of simulated flight.* 

Instructor Tulorlal provides SI with self-paced programmed instruction In 
the capabilities and use of the flight simulator. * 
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Table 1. (Contlp.ied) 



TRAINING MANAGEMENT FEATURES 

T otal System Freeze permits SI to suspend simulated flight by freezing 
a11 system parameters.' 

Reset permits SI to return the simulated aircraft to a stored set of 
conditions and parameters.' 

Crash gnd/or Kill Override permits SI to allow simulated flight to 
continue without Interruption following a "crash" or "kill.*' 

Automated Adaptive Training Is the computer- control led variation In 
task difficulty^ complexity^ and/or sequence based on student's 

performance.^ 

Programmed Miss i on Scenarios are computer-controlled standardized 

a 

training missions based on pre-programmed event sequences. 

VARIATION OF TASK DIFFICULTY FEATURES 

Automated Malfunction Insertion permits SI to preprogram a sequence of 
aircraft component malfunctions and/or emergency conditions.* 

Environmental permits SI to vary environmental conditions such as wind 
direction and velocity, turbulence, temperature, and visibility. 

Dyr ^ cs permits SI to vary flight dynamics characteristics such as 
stability, system gain, cross-coupling, etc. 

Motion permits SI to provide a student with platform motion system 
cues such as roll, pitch, lateral, and verticals* 



Table 1. (Continued) 



Flight Syston Freeze permits SI simultaneously freeze flight control 
and propulsion systems, position, altitude, and heading. 

Position Freeze permits SI to simultaneously freeze latitude and 
longitude. 

Attitude Freeze permits SI to simultaneously freeze pitch, bank, and 
heading. 

Parameter Freeze permits SI to freeze any one or a combination of 
flight parameters.* 

INSTRUCTOR MONITOR AND FEEDBACK FEATURES 

Closed Circuit TV permits SI to monitor student's behavior from the 
Instructor console. 

Repeaters /Annunc 1 ator s provide SI with replicas or analog 

representations of flight Instruments and controls at che Instructor 
console. 

Instructor Console Displays permit SI to view alphanumeric and/or graphic 
CRT displays of performance data at the Instructor console.* 

Automated Performance Alert provides SI with visual and auditory signals 
that Indicate specific performance deficiencies. 

STUDENT FEEDBACK FEATURES 



Record/Playback permits SI to record and subsequently play back a segment 
of simulated flight.* 



Table 1. (Concluded) 



Automated Performance Feedback provides a student with visual and 
auditory signals (including verbal messages) that identify 
performance deficiencies. 

Automated Voice Controller is the computer-based technology that 

simulates the role of a controller by combining speech generation, 
speech recognition, and situation awareness capabilities. 

Hard Copy provides a record of alphanumeric and/or graphic 

a 

performance data from the automated performance measurement system. 



* These features were included in the Phase II questionnaire. 
(See Appendix.) 



It appears that military ATDs are more often treated as substitute 
aircraft than as teaching machines. A recent report by the United States 
General Accounting Office (1983) concluded that the Armed Services have not 
sufficiently analyzed their training requirements for simulators. Nor have 
they adequately incorporated simulators into their training programs. In 
justifying the purchase of ATDs, the Services have focussed instead on 
*dupli eating the actual weapon systems and their surroundings . . .with 
little reference to how the devices could meet training needs" (p. 4). 

The present investigation was conducted at the request of the 
Simulator System Program Office (SimSPO) of the Air Force Systems Command, 
Aeronautical Systems Division (AFSC/ASD). The objectives of this 
investigation were: 

1. To document and compare the utilization (i.e., frequency and ease 
of use) of AIFs. 

2. To document and compare the utility (i.e., training value) of 
AIFs. 

3. To compare the utility and utilization patterns of AIFs in 
replacement (e.g., basic, primary, lead-in, initial) and 
continuation (e.g., advanced, follow-on, refresher, operational) 
training units. 



A broader objective of this Investigation was to provide a database that 
could be Lelpful both In defining the requirements for ATD procurements and 
in developing future ATD training programs. 

These ob^^ctives will be accomplished in several phases by means of a 
survey of simulator ii^structors (Sis) from the Air Force Major Commands 
(MAJCOMs). Phase I has already been completed, and the results of that 
survey are documented in an earlier report (Polzella, 1983). 

The subjects in Phase I were 134 simulator-qualified instructor pilots 
(IPs) and weapons director instructors (WDIs) assigned to replacement 
training units (RTUs) and continuation training units (CTUs) at F-4E, F-4S, 
F-15, A-10, and E-3A Tactical Air Command (TAC) training sites. The 
results indicated that most TAC Sis received little training in the use of 
AIFs and that most features were not used very often. Several factors 
appeared to have contributed to the low usages: (o) hardware and/or 
software unreliability, (b) time-consuming implementation, (c) functional 
limitations, and (d) design deficiencies. The results of a multiple 
regression analysis indicated that ease of use and training value accounted 
for most of the variability in the frequency-of-use ratings. 

The utility and utilization of particular AIFs differed both as a 
function of ATD and of training unit. For example, features such as freeze 
and reset were generally used more often during RTU missions, whereas 
progroinmed mission scenarios were generally used more often during CTU 
missions. These differences appeared to reflect differences in the 
respective training missions. Thus, RTU missions characteristically 
Include a series of discrete procedural exercises, whereas lengthier 
scenarios are common during CTU missions. 

Based on the results of Phase I, it was recommended that future 
procurement of AIFs be preceded by a detailed front end analysis that 
clearly relates AIF capability to training needs as well as to any major 
constraints in the operational environment. During procurement, AIF 
specifications should be prepared so as to meet user needs and ensure 
equipment reliability. After operational deployment, the user should 
provide adequate instructor/operator training in AIF use. 

Phase II, which is described in this report, extended the survey to 
Sis from Air Training Command (ATC), Military Airlift Command (MAC), and 
Strategic Air Commend (SAC). 



II. METHOD 



Subjects 

The subjects in Phase II were 273 instructor pilots (IPs), instructor 
flight engineers (IFEs), and instructor radar navigators (IRNs) assigned to 
RTUs or CTJs at the following training sites: Williams AFB (T.50/T-37 and 
T-51/T-38; RTU), Altus AFB (C-5A and C-141; RTU), Dover AFB (C-5A; CTU), 
McGuire AFB (C-141; CTU), Little Rock AFB (C-130; RTU, CTU), Kirtland AFB 
(CH-3 anri HH-53; RTU, CTU), Plattsburgh AFB(FB-111A; RTU, CTU), and Pease 



AFB (FB-lllA; CTU). The distribution of Sis among the various ATDs and 
training units Is shown In Table 2. The Sis' hours of experience as 
instructors are summarized In Table 3. 



Table 2. The Number of Sis Surveyed In Phase II 









Training unit 


ComRiand 


ATD 


Type of SI 


RTU 


CTU 


ATC 


T-50 


IP 


29 






T-51 


IP 


21 




MAC 


C-5A 


!P 


16 


11 






IFE 


13 


6 




C-141 


IP 


26 


6 






IFE 


14 


7 




C-130 


IP 


13 


7 






IFE 


8 


6 




CH-3 


IP 


3 


4 






IFE 


3 


1 




HH-53 


IP 


5 


6 






IFE 


0 


6 


SAC 


FB-lllA 


IP 


6 


24 






IRN 


n 

168 


21 
105 
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Table 3. Simulator Instructor's Mean (and 
Standard Deviation) Number of 
Instructor Hours 



Conmand 


ATD 


RTU 


CTU 


ATC 


T-Bl 


1 / <}»0 

(169.5) 
(96.8) 




MAC 


C-5A 


511 1 
(452.1) 


454 1 
(386.8) 






(531.4) 


1 1 / H» U 

(1504.1) 




C-130 


126.8 
( 80.4) 


419.8 
(189.9) 




CH-3 


169.2 
(94.4) 


263.4 
(132.7) 




HH-53 


139.2 
(149.8) 


482.5 
(452.1) 


SAC 


FB-lllA 


797.1 
(693.3) 


353.0 
(818.4) 



Questionnaire 

The questionnaire that was used to survey the Instructors is shown in 
the Appendix. It Is similar to that used In Phase I. (See Polzella, 1983, 
Appendix A.) The questionnaire requested background Information (I.e., 
flying and simulator experience), a brief description of a typical training 
mission, and Included a list of 16 AIFs (drawn from the list In Table 1) 
and their definitions and five questions concerning the utility and 
utilization of each feature: 



1. How often have you used It? 

2. How easy Is It to use? 

3. How much training did you receive In Its use? 

4. What Is Its training value? 

5. What Is Its potential training value? 
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For the fifth question, Sis were asked to assume that they had no prior 
knowledge of the features and to base their responses on the feature 
definitions alone. This question was Included In order to achieve a common 
basis for comparison among all Sis. This was not otherwise possible 
because the various ATOs were not similarly equipped. 

Responses to each question were Indicated by checking the appropriate 
interval along a seven-point, successive-category rating scale. (The 
scales for questions 2 and 4 Included a zero-point Interval for Indicating 
"no basis for judgment.") The Intervals of each scale were labeled with 
descriptive adjectives In order to facilitate responding and to help 
Interpret the ratings. Additional space was provided for comments. 

Procedure 

The questionnaire was administered on-site to various sized (N « 2 to 
25) groups of Sis. The Sis were briefed on the purpose of the 
Investigation and copies of the questionnaire were distributed and 
thoroughly reviewed prior to being filled out. The questionnaire could be 
completed In approximately 30 minutes. 



III. RESULTS 

Table 4 lists the 16 AIFs that were Included In the questionnaire 
along with their definitions. The table also Indicates a mnemonic code for 
each feature, which will be used In subsequent tables. The AIF 
capabilities of the various ATDs are shown In Table 5. 



Table 4 . Advanced Instructional Features Included 
In the Phase II Questionnaire 



Code Feature 

IPX Instructor Pilot Tutorial - provides the IP with self- 
paced programmed instruction In the capabilities and use 
of the flight simulator. 

R Reset - permits Instructor to "return" the simulated 
aircraft to a stored set of conditions and parameters. 

TSF Total System Freeze - permits Instructor to Interrupt and 
suspend simulated flight by freezing all system 
parameters. 

RB Recorded Briefing - permits Instructor to provide student 
with mrormati on about a structured training session 
through audio/visual media presentation. 

D Demonstration - permits Instructor to demonstrate 

aircraft maneuver (s) by prerecording and subsequently 
playing back a standardized segment of simulated flight. 

8 
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Table 4. (Concluded) 
code Feature 



RP Record/Playback - permits Instructor to record and 

suDsequentiy playback all events that occurred during a 
segment of simulated flight. 

E Environmental - permits Instructor to vary environmental 
conditions such as wind direction and velocity, 
turbulence, temperature, visibility, etc. 

AMI Automated Malfunction Insertion - permits Instructor to 
preprogram a sequence of aircraft component malfunctions 
and/or emergency conditions. 

PF Partial Freeze - permits Instructor to freeze various 
Tiignt parameters or parameter combinations such as 
altitude, heading, position, attitude, flight system, 
etc. 

CKO Crash a"d/or Kill Override - permits Instructor to allow 
simulated f iignt to continue without Interruption 
following a "crash" or "kill." 

M Motion - permits Instructor to vary platform motion 

system cues such as roll, pitch, lateral, vertical, etc. 

HC Hard Copy - provides a record of alphanumeric and/or 

grapnic performance data from the automated performance 
measurement system for debriefing purposes. 

AAT Automated Adaptive Training - computer- control led 

variations in task difficulty, compisxlty, and sequence 
based on pilot's performance. 

PMS Programmed Mission Scenarios - computer-controlled 

standardized training sessions based on pre-programmed 
event sequences. 

PRM Procedures Monitoring - permits Instructor to monitor 
aiscrete actions performed by the student In accordance 
with a procedurally defined checklist. 

PAM Parameters Monitoring - permits Instructor to monitor 
various instrument readings, control settings, aircraft 
states, or navigational profiles. 
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Table 5. AIF Capability of Each ATD 



Feature T-50 T-51 C-5A C-141 C-130 CH-3 HH-53 FB-lllA 



TDT 

Ir 1 


















n 
R 


Y 
A 


Y 
A 


Y 
A 


A 


Y 

A 


X 


X 


X 


lor 


y 
A 


Y 
A 


Y 
A 


Y 


X 


X 


X 


X 


RB 


















n 

V 










X 


X 


X 


X 


DO 

Kr 


Y 
A 


Y 
A 






V 

A 


X 


X 


X 


c 
C 


Y 
A 


Y 
A 


Y 
A 


Y 
A 


Y 

A 


X 

A 


X 


X 


AMT 


Y 
A 


Y 
A 






X 








PF 


X 


V 

A 


V 
A 


Y 

A 


Y 
A 


Y 
A 


A 


V 
A 


CKO 


X 


X 


X 


X 


X 


X 


X 


X 


H 


X 


X 


X 


X 


X 


X 


X 


X 


HC 










X 






X 


AAT 


















PHS 










X 






X 


PRM 










X 






X 


PAM 










X 


X 


X 


X 



The raw data Included the Sis' descriptions of a typical training 
mission and their ratings on each question coded as 0 (no basis for 
jud.inent) to 7 (maximum rating). The ratings were classified by ATD (T-50, 
T-51. C-5A. C-141. C-130, CH-3. HH-53. FB-illA). training unit (RTU. CTU), 
question (1 through 5) a nd AIF (1 th rough 16). The resulting data matrix 
was unbalanced due to the differences in the number of Sis and in the AIF 
capabilities of the various ATDs (see Tables 2 and 5). This necessitated 
analyzing the data from each ATD separately, with two exceptions. The C-5A 
and C-141 data and the CH-3 and HH-53 data were pooled, respectively, 
before they were analyzed. The pooling of these data was not inappropriate 
since the respective training missions were highly similar. Moreover, the 
r-esulting increase in sample size was statistically desirable, especially 
in the case of the CH-3 and HH-53 data. 
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A1r Training Cofomand 



T'50/T-51 Simulator Training Missions 

The T-50 (T-37) and T-51 (T-38) training missions are similar, each 
lasting approximately 1-1/2 hours. In addition, there are briefing and 
debriefing periods lasting 20 to 30 minutes each. Although missions may 
vary from partial task to full mission profile, most missions Include 
takeoff s. Instrument and emergency procedures, turns, climbs, descents, 
course Intercepts, penetrations, and multiple approaches. In addition, the 
T-51 mission emphasizes certain advanced skills that reflect the particular 
characteristics of the T-38 aircraft, such as steep turns, unusual 
attitudes, and vertical s maneuvers. 

Frequency of AIF Use 

The frequency- of- use ratings are summarized In Table 6. Here and In 
subsequent tables, the features are listed In decreasing order according to 
the mean ratings. The Individual ratings ranged from 1 (never use) to 7 
(use roost often). Thejfrequency of AIF use appears to be fairly high, with 
most of the means ranging Trom 3.5 to 6.5 (I.e., moderately often to very 
frequently ). — * 



Table 6. T-50, T-51 Instrument Flight Simulators: 
Mean Ratings (and Standard Deviations) 
of the Frequency of AIF Use 



Feature 


T-50 


T-bl Combined 


M 


6.6 


6.2 


6.T 






(1.2) 


(1.9) 






TSF 


5.7 


6.1 


5.9 






(1.4) 


(0.9) 






R 


5.6 


6.2 


5.9 






(1.3) 


(0.8) 






E 


5.5 


6.3 


5.8 






(1.3) 


(0.7) 






**PF 


3.9 


5.3 


4.5 




(1.7) 


(1.4) 




CKO 


3.6 


3.8 


3.7 




(1.4) 


(1.4) 




RP 


3.4 


3.9 


3.6 




(1.3) 


(1.3) 




AMI 


3.7 


3.1 


3.5 




(2.2) 


(1.9) 




*Comb1ned 


4.8 


5.1 





*p < .05. 
**£< .01. 
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A two- factor (ATD x AIF) repeated measures analysis of covariance 
(covariate = number of instructor-hours) was used to analyze the data. The 
analysis indicated that the r-51 Sis overall mean rating (4.9) was 
significantly higher than that of the T-50 Sis (4.5), F(l,47) = 5.26, p < 
.05. There was also a significant main effect of AIF, F( 7,336) = 42. 9Z, p 
< .001, and a significant AIF by ATD interaction, F(7,3J6) = 3.17, £ < .OT. 

The significant main effect of AIF implies that there are 
significant differences among the overall ratings of each feature. The 
Tukey honestly significant difference (HSD) test (Keppel, 1973, p. 138) was 
used to determine these differences. The results of the test are shown in 
Table 6 by the placement of brackets around those means that did not differ 
significantly (£ > .01). Thus, motion, total system freeze, reset, and 
environmental received the highest ratings, whereas the remaining features 
received significantly lower ratings. 

The significant interaction implies that the pattern of ratings was 
different for the two groups of Sis. The Dunn test (Keppel, 1973, pp. 147- 
149) was used to determine the locus of this interaction by making post hoc 
comparisons between the various T-50/T-51 adjusted mean pairs. The 
significant comparisons are indicated In the table by asterisks. Only one 
significant difference was obtained, that for partial freeze, which was 
rated significantly higher by T-51 Sis (£ < .01). 

Ease of AIF Use 

The ease-of-use ratings are summarized in Table 7. The individual 
ratings ranged from 1 (most difficult) to 7 (easiest). Ease of use appears 
to be very high, with most of the means ranging from 4.5 to 6.5 (i.e., easy 
to very easy ). Unlike the frequency of use ratings, the means in Table 7 
were based on a variable frequency that reflected the number of Sis who 
actually used each feature. Consequently, the nonpar ametric Wilcoxen test 
was used to compare the T-50 and T-51 ratings, overall, and t-tests were 
used to make separate T-50/T-51 comparisons for each feature. The 
significant differences are indicated in Table 7 by asterisks. Thus, the 
T-51 mean rating of partial freeze was significantly higher than the 
corresponding T-50 mean rating, t(47} = -2.84, £ < .01. 

Training in AIF Use 

The amount of training received in AIF use is summarized in Table 8. 
The individual ratings ranged from 1 (no training received) to 7 (greatest 
amount received). The mean ratings were fairly high, although not as high 
as the frequency- of- use and ease-of-use ratings. Most ranged from 3.5 to 
5.5 (i.e., moderate to considerable) . 

A two-factor (ATD x AIF) repeated measures analysis of covariance 
(covariate = number of Instructor-hours) was used to analyze the data. The 
analysis revealed significant main effects of ATD, F(l,47) = 5.84, £ < 
.025, and of AIF, F(7,336) « 11.86, £ < .001. The interaction was not 
significant, F(7,336) = i.71,£ > .05. If appears that T-51 Sis received 
more training'in the use of ATFs than did T-50 Sis; however, the pattern of 
use for the two groups was statistically equivalent. The Tukey HSD test 
was used to determine the significant differences among the combined 
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Table 7. T-50, T-51 Instrument Flight Simulators: 
Mean Ratings (and Standard Deviations) 
of the Ease of AIF Use 



Feature 


— f cK 

T-50 


T-51 


Combined 


TSF 


6.7 


6.8 


6.7 




(0.6) 


(0.4) 




N 


6.4 


6.4 


6.4 




(0.6) 


(0.7) 




R 


5.9 


6.0 


6.0 




(0.8) 


(0.7) 




*PF 


5.5 


6.4 


5.9 




(1.2) 


(0.7) 




CKO 


5.7 


5.8 


5.7 




(1.2) 


(1.3) 




E 


5.2 


5.1 


5.2 




(1.3) 


(l.c) 




AHI 


5.2 


5.3 


5.2 




(1.7) 


(1.0) 




RP 


4.9 


5.4 


5.1 




(1.0) 


(1.1) 




Unweighted Means 


5.7 


5.9 


5.8 



*2.< .01. 



ratings of each AIF, and the results of that test are shown in Table 8 by 
the placement of brackets around the means that did not differ 
significantly (g > .Olj. The results closely parallel those shown In Table 
6. Thus, the features receiving the greatest amount of training were also 
those features that were most frequently used. 
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Table 8. T-50, T-51 Instrument Flight Simulators: Mean 
Ratings (and Standard Deviations) of the Amount 
of Training Received In AIF Use 



Feature 


T-50 


T-51 ComDined 


"TTT 




5.6 


5.0- 






(1.7) 


(1.3) 






M 

Pi 


4.6 


4.8 


4.7 






(1.6) 


(2.0) 






D 


4.2 


5.1 


4.T 






(1.1) 


(0.9) 








c 

b 


4 2 


4.7 




4._4_ 






(1.6) 


(1.1) 








RP 


3.5 


4.6 




4.0 




(1.2) 


(0.9) 






PF 


i.3 


4.5 


3.8 




(1.3) 


(1.2) 




CKO 


3.7 


4.0 


3.8 




(1.3) 


(1.3) 




AMI 


3.3 


3.5 


3.4 




(1.7) 


(1.6) 






Combined** 


3.9 


4.6 


4.2 



< .025. 



Training Value of AIFs 

The training value ratings are summarized In Table 9. The Individual 
ratings ranged from 1 (no training value) to 7 (greatest training value). 
The ratings were extremely high, with most of the means In the 4.5 to 6.5 
range (I.e., considerable to great ) . Like the ease-of-use ratings, the 
means In Table 9 were based on a variable frequency that reflected the 
number of Sis who actually used each feature. Consequently, the 
nonparametrlc VHIcoxen test was used to compare the T-50 and T-51 ratings, 
overall, and t-tests were used to make separate T-50/T-51 comparisons for 
each feature. There were no significant comparisons (£ > .05). 
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Table 9. T-50» T-51 Instrument Flight Simulators: 
Mean Ratings (and Standard Deviations) 
of the Training Value of AIFs 



Feature 


T-50 


T-51 


combined 


E 


6.2 


6.Z 


6.Z 




(0.9) 


(1.0) 




N 


6.1 


6.0 


6.1 




(1.0) 


(1.0) 




R 


5.9 


6.3 


6.1 




(0.9) 


(0.8) 


TSF 


6.0 


6.1 


6.0 




(0.8) 


(1.0) 


RP 


4.9 


5.5 


5.1 




(1.0) 


(1.3) 


PF 


4.8 


5.3 


5.0 




(1.4) 


(1.1) 


AMI 


5.0 


4.1 


4.6 




(1.6) 


(2.1) 


CKO 


4.4 


4.4 


4.4 




(1.5) 


(1.5) 




Unweighted Means 


5.4 


5.5 


5.4 



Potential Training Value of AIFs 

The potential training value ratings are summarized in Table 10. The 
individual ratings ranged from 1 (no potential value) to 7 (greatest 
potential value). Like the training value ratings, the potential training 
value ratings were very high, with most means ranging from 4.5 to 6.5 
(i.e., considerable to great ). 

A two-factor (ATD x AIF) repeated measures analysis of covariance 
(coyariate « number of instructor-hours) was use^i to analyze the data. The 
analysis revealed a significant main effect of AIF, F(15.720) « 16.56. p < 
.001, and a significant ATD by AIF interaction, F(157720) . 1.84, p < .05. 
The main effect of ATD was not significant, F < T.OO. 

The Tukey HSD test was used to determine the significant differences 
among the combined ratings of each AIF, and the results of that test are 
shown in Table 10 by the placement of brackets around the means that did 
not differ significantly (£ > .01). Once again the results closely 
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Tabl e 10. T-bO, T-51 Instrument Flight Simulators: 

Mean Ratings (and Standard Deviations) 

of the Potential Training Value of AIFs 



Feature 


T-50 


T-51 


Combined 


E 




6.3 


6.3 




(1 01 


(1.0) 






R 


6.3 


6.2 


6.3 








(0.7) 






H 


6.3 


5.9 


6.2 






(1 11 


(1.1) 






TSF 


6.1 


6.2 


6.1 






fl 11 


(0.8) 






RP 


5 2 


5.9 


5.5 






f 1 '^l 


(1.2) 








PF 


5.0 


S S 




T.2 






(1 71 


(0.8) 






AMI 


5«4 


4 7 


5.1 






f 1 (\l 


(1.8) 






PAM 


4.6 


5.5 


5.T 








(1.4) 






D 


4.6 


5.5 


4.9 








(1.2) 






CKO 


4.8 


4.9 


4.8 






r 1 A\ 


(1.1) 






RB 


4.7 


5.0 


4.8 






(9 ni 


(1.5) 






PHS 


4.8 


4.3 


4.6 






fl ftl 


(1.0) 






PRM 


4.0 


5.0 


4.6 






(2 01 


(1.5) 






HC 


4.5 


4.5 


4.5 






(2.0) 


(1.3) 








AAT 


4.1 


4.0 


4.1 






(2.0) 


(1-1) 






IPT 


4.0 


4.4 


4.1 






li-ii 


(1.5) 




Combined 


5.1 


5.2 


5.1 
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parallel those shown In Table 6. Thus, the features rated highest in 
potential training value were also those features that were most frequently 
used. 

Despite the Indication of a significant Interaction, Its locus was not 
revealed by the Dunn test, which was used to make post hoc comparisons 
between the various T-50/T-51 adjusted mean pairs. The Interaction was, 
therefore, considered to be of minimal Importance. 

Interrelations Among the Variables 

Table 11 shows the Intercor relations among the ratings of each feature 
on each of the five questions. A11 the coefficients are positive and 
significant, £ < .001. Therefore, a feature's rating on any question can 
be predicted with greater than chance accuracy alven Its ratlna on any 
other question. For example. It can be generally stated that the more 
frequently a feature was used, the easier It was to use, the more training 
was received In Its use, and the greater Its training and potential 
training value. 

Table 12 sunwarlzes the results of a stepwise multiple linear 
regression analysis In which the frequency of AIF use was predicted from a 
linear combination of the remaining variables. The table Indicates that, 
together, the predictor variables accounted for 53 percent of the variance 
In the frequency-of-use ratings, the most Important predictor being 
training value followed. In order, by ease of use, potential training 
value, and training received. 



Military Airlift Command 



The MAC ATDs that were surveyed Included the C-5A, C-141, C-130, CH-3, 
and HH-53. The same analyses that were performed on the ATC data were used 
to analyze the MAC data. As discussed previously, C-5A and C-141 and the 
CH-3 and HH-53 data were pooled, respectively, before they were analyzed. 

C-5A/C-141 Flight Simulators 

Training mission . The C-5A/C-141 RTU and -TU training missions are 
similar, each lasting approximately 4 hours. Usually, students will "swap 
seats" after 2 hours. In addition, there are 2-hour briefing and 1-hour 
debriefing sessions. Both missions consist of a wide variety of normal and 
emergency procedures that stress crew coordination. Partial or total real- 
time scenarios typically provide the training context, especially for the 
CTU mission. 

Frequency AIF use . The frequency-of-use ratings are summarized In 
Table 13. Like the ATC ratings, the C-5A/C-141 ratings were fairly high, 
with most of the means In the 3.5 to 5.5 range (I.e., moderately often to 
frequently ) . A two-factor (training unit x AIF) repeatea measures analysis 
or covanance (covarlate ■ number of Instructor-hours) revealed a 
significant main effect of AIF, F(5,465) ■ 23.53, p < .001, and a 
significant AIF by training unlt'lnteractlon, F(5,465) ■ 4.86, £ < .001. 
The main effect of training unit was not sIgnlTlcant, F(l,92) < 1.00. 
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Table 11 . T-50. T-51 Instrument Flight Simulators: 

Matrix of Intercorrelations Among Frequency 
of Use, Ease of Use, Training Received, 
Training Value, and Potential Training Value 





FREQUSE 


EASEUSE 


TRECb 


TVALUE 


PTVALUE 


FREQUSE 


1.00 










EASEUSE 


.51 


1.00 








TRECO 


.49 


.38 


1.00 






TVALUE 


.63 


.43 


.42 


1.00 




PTVALUE 


.60 


.35 


.38 


.72 


1.00 



Note All correlations are significant, £ < .001. 



Table 12. T-50, T-51 Instrument Flight Simulators: 
Multiple Linear Regression of Frequency 
of AIF Use on Ease of Use, Training 
Received, Training Va'fue, and 
Potential Training Valuct 



Dependent Variable: 



Frequency of AIF Use" 



Multiple R: 
Multiple R-Square: 
Standard Error of Estimate: 



.73 
.53 
1.29 



Analysis of Variance: 



Regression 
Residual 



Sum of Squares 

835.3630 
741.6020 



DF Mean Square 



4 

444 



208.8408 
1.6703 



F 

125.03 



£ 

< .001 



Summary of Stepwise Regression: 







Multiple 
R 


Increase 


F-to- 


Step No. 


Variable 


in RSQ 


Enter 


1 


TVALUE 


.63 .39 


.39 


288.02 


2 


EASEUSE 


.68 .46 


.07 


57.33 


3 


PTVALUE 


.71 .50 


.04 


35.97 


4 


TRECO 


.73 .53 


.03 


26.72 
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Table 13. C/5A-C-1 41 Flight Simulators: 
Mean Ratings (and Standard 
Deviations) of the Frequency 
of AIF Use 



Feature 



"cnr 



Combined 



N 

CKO* 

E 

R 

PF** 

TSF 
Combined 



5.1 
(2.0) 

5.5 
(2.1) 

5.1 
(1.6) 

4.5 
(1.9) 

4.3 

(1.8) 

3.1 
(1.4) 

4.6 



5.7 
(2.1) 

4.4 

(2.2) 

5.3 
(1.7) 

4.3 
(1.9) 

3.1 

(2.1) 

3.4 

(1.5) 

4.4 



5.3 
5.2 
5.£ 
T.4 

4.5 



*E. < .05. 
**fi. < .01. 



A Tukey HSD test revealed that the highest ratings were assigned to 
motion, crash/kill override, and environmental. A Dunn test indicated that 
the RTU Sis assigned significantly higher freauency-of-use ratings* to 
crash/kill override and to partial freeze than did the CTU Sis. 

Ease of AIF use . The ease-of-use ratings are summarized In Table 14. 
Ease Of use appeared to be extremely high, with most of the means ranging 
from 5.5 to 6.5 (I.e., very easy ). There was no overall difference between 
the RTU and CTU ratings, T » 7, £ > .05, nor were there any significant 
RTU/CTU differences for particular features. 

Training In AIF Use . The amount of training received In AIF use Is 
summarized in Table 15. Host of the means range from 3.5 to 4.0 (I.e., 
moderate ) and are somewhat lower than the corresponding ATC means. The 
analysis of covarlance revealed a significant effect of AIF, F(5,465) > 
3.49, £ < .01. Sis apparently recelyed most training In the use of 
environmental, reset, and motion. Interestingly, thtst AIFs were rated 
'owest In ease of use. Neither the main effect of training unit, F < 1.00, 
h. * the AIF by training unit Interaction, F(4,465) • 1.27, p > .OSTwas 
significant. 
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Table 14. C-5A/C-141 Flight Simulators: Mean Ratings 
(and Standard Deviations) of the Ease of 
AIF Use 



Feature 


KIU 


PTB 


vUlllU 1 llwU 


CKO 


6.3 


6.1 


6.2 




(0.9) 


(1.0) 




TSF 


6.0 


5.9 


6.0 




(0.9) 


(0.9) 




PF 


5.9 


5.7 


5.8 




(0.9) 


(0.9) 




M 


5.7 


5.9 


5.8 




(1.0) 


(1.3) 




R 


5.4 


5.5 


5.5 




(0.8) 


(1.0) 




E 


5.3 


5.2 


5.3 




(0.8) 


(0.8) 




Unweighted Means 


5.8 


5.7 


5.8 



Table i5. C-5A/C-141 Flight Simulators: Mean Ratings 
(and Standard Deviations) of the Amount of 
Training Received in AIF Use 



Feature 


RTU 


CTU Combined 


E 


4.2 


3.7 


4.1 






(1.3) 


(1.6) 






R 


3.7 


3.6 


T.7 






(1.7) 


(1.8) 






M 


3.6 


3.8 


3.2 






(1.5) 


(1.6) 






TSF 


3.7 


3.6 


3.6 




(1.5) 


(1.5) 




CKO 


3.6 


3.6 


3.6 




(1.6) 


(1.5) 




PF 


3.5 


3.2 


2.4 




(1.5) 


(L9i 




Combined 


3.7 


3.6 


3.7 



Training value of AIFs. The training value ratings are summarized in 
Table W, ine ratings were quite high, with most of the means in the 4.5 
to 5.5 range (i.e., considerable ) . While there was no overall difference 
between the RTU and CTU ratinas, T = 2.5, p > .05, the CTU Sis assigned 
significantly higher ratings to moTion and to total system freeze than did 
the RTU Sis, t(91) = -2.38, £ < .05, and t(93) = -2.51, £ < .05. 
respectively." ~ 



Table 16. C-5A/C-141 Flight Simulators: Mean Ratings 
(and Standard Deviations) of the Training 
Value of AIFs 



Feature 



RTU 



"CTir 



Combined 



M* 5.4 

(1.3) 

R 5.2 
(1.2) 

E 5.2 
(1.3) 

PF 5.2 
(1.4) 

TSF* 4.4 
(1.4) 

CKO 4.6 
(1.7) 

Unweighted Means 5.0 



6.1 

(1.1) 

5.5 
(1.1) 

5.3 
(1.3) 

5.0 
(1.0) 

5.2 
(1.3) 

4.8 

(1.7) 

5.3 



5.6 
5.3 
5.2 
5.1 
4.7 
4.7 
5.1 



*£ < .05. 



C-130 Flight Simulator 

Training mission . The C-130 RTU and CTU missions are similar to those 
of the C-5A/C-141. tach training session consists of a 1- to 2-hour 
prebriefing, a 4-hour mission, and a 1-hour debriefing. Both the RTU and 
CTU student crews receive extensive practice in normal and emergency 
procedures during takeoff s, in-flight malfunctions, and full-stop landings. 
CTU cross-country missions are typically flown under a variety of 
passenger/ cargo configurations and weather conditions. 

Frequency of AIF Use . The C-130 frequency-of-use ratings are 
summanzea in i able 17. Although the overall mean rating of 3.9 (i.e., 
moderately often ) was somewhat lower than that of the other ATDs, the 
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Table 17. C-130 Flight Simulator: Mean Ratings 
(and Standard Deviations) of the 
Frequency of AIF Use 



Feature 


RTU 


CTU Combined 


M 


6.6 


7.0 


6.7 




(1.4) 


(0.0) 




E 


5.4 


6.4 


?.8 




(1.0) 


(0.9) 




R 


5.7 


5.5 


5.TI 




(1.0) 


(1.3) 




CKO 


5.1 


5.0 


[5.1 




(1.4) 


(1.4) 




TSF 


4.7 


4.6 


4.Tj 




(1.3) 


(1.6) 




PF 


4.7 


4.1 


■T.4 




(1.5) 


(1.4) 




PAM 


4.2 


3.2 


3.TJI 




(1.7) 


(1.9) 




AMI** 


3.9 


1.5 f 


'T.o 




(2.8) 


(0.5) * 




PRM 


3.0 


2.8 


2.TJ 




(1.9) 


(1.4) 




HC 


2.6 


2.1 


2.4 




(1.2) 


(0.8) 




D 


1.6 


2.4 


1.9 




(1.0) 


(1.6) 




RP 


1.9 


2.0 


1.9 




(1.2) 


(0.7) 




PMS 


2.0 


1.5 


1.8 




(1.9) 


(0.7) 




Combined 


3.9 


3.7 


3.9 



**p < .01. 



highest rated features were nevertheless used very frequently. The 
analysis of covarlance revealed a significant effect of AIF, F( 12,384) « 
53.23, p < .001, and a islgnlf leant AIF by training unit Interaction, 
F(12,384) n 3.25, £ < .001. The difference between the overall RTU and CTU 
ratings was not significant however, F < 1.00. 
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The Tukey HSD test indicated that the highest ratings were assigned to 
motion, environmental, reset, and crash/kill override. These same features 
received consistently high ratinas at the other sites as well. Procedures 
monitoring, hard copy, demonstration, record/playback, and programmed 
mission scenarios received the lowest ratings. The Dunn test revealed only 
one significant RTU/CTU comparison, that for automated malfunction 
Insertion; it was used significantly more often by RTU Sis. 

Ease of AIF Use . The ease-of-use ratings are summarized in Table 18. 
As was the case for all the ATDs surveyed, ease of use was quite high, with 
most of the C-130 means in the 4.5 to 6.0 range (i.e., easy to very easy ) . 
There was no overall difference between the RTU and CTU ratings, T « Z2.5, 
£ > .05; however, parameters monitoring was rated significantly easier to 
use by RTU than by CTU Sis, t(32) - 2.72, £ < .05. 

Training in AIF use . The amount of training received in AIF use is 
summanzea in laoie is. Most of the means are in the moderate range. The 
analysis of covariance revealed a significant effect of AIF, F(12,384) » 
15.03, £ < .001, but neither the main effect of training unit~nor the 
interaction was significant, Fs < 1.00. The results of the Tukey HSD test 
closely paralleled the results obtained from the frequency- o*- use data. 
Those features that received the most and least amounts of training were 
used most and least often, respectively. 

Training value of AIFs . The training value ratings are summarized in 
Table Tyl ine ratings were fairly high, with most of the means in the 4.0 
to 6.0 range (i.e., moderate to great ). The lowest ratings were generally 
assigned to the least frequently used features. The overall difference 
between the RTU and CTU ratings was not significant, T = 22, £^> .05; 
however, automated malfunction insertion was rated higher by RTU than by 
CTU Sis, t(24) . 3.13, £ < .01. 

CH-3/HH-53 Flight Simulators 

Training mission . The CH-3/HH-53 training mission lasts approximately 
3 to 4 hours and includes two separate 1-1/2- to 2-hour missions; in 
addition there are prebriefings and debriefings. Due to the lack of a 
visual system, instrument flying is emphasized. Student crews practice 
takeoffs, holding patterns, fix-to-fix, blind landings, and emergency 
procedures. Malfunctions, which are inserted manually and with great 
regularity, typically require the crew to respond in a highly coordinated 
manner. Although the RTU and CTU missions are similar, the CTU mission 
tends to be somewhat more demanding. 

Frequency of AIF use . The CH-3/HH-53 frequency-of-use ratings are 
sumnarizea in rable zl. The frequency of AIF use appears to be fairly 
high, with most of the means in the 4.5 to 6.0 range (i.e., frequently to 
very frequently ). Only demons t. at ion was rarely used. The analysis of 
covariance revealed that, overall, the CTU Sis reported greater AIF use 
than did the RTU Sis, F(l,25) • 9.88, £ < .01. There was also a 
significant main effect of AIF, F(8,208) « 30.76, £ < .001. The AIF by 
training unit interaction was nol significant, F(8,208) « 1.29, p > .05. 
The Tukey HSD test revealed that, except for record/ pi ay back and^ 
demonstration, the frequency of AIF use was uniformly high. 
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Table 18. C-130 Flight Simulator: Mean Ratings 
(and Standard Deviations) of the Ease 
of AIF Use 



Feature 


RTU 


CTU 


comDineo 


TSF 


6.2 


6.3 


o.z 






fn Qi 




CKO 


6.0 


6.3 


C 1 

0.1 






(0 91 




M 


5.8 




C\ Q 

0.7 




(1 11 


(1 01 




DC 

rr 




C\ ft 






(1.21 


(1.1) 




n 
K 










(1.01 






c 
c 


0.4 


R Q 


0.0 




(1 21 


(1.01 




AMI 


c n 
9.0 




0.0 




(1 9) 


(0.91 




PMS 


4.7 




o.u 




(1 11 


(1 51 




KAM* 


9. J 


O.O 


A 7 




(1.3) 


(1.7) 




HC 


4.7 


4.8 


4.7 




. (1.4) 


(1.6) 




PRH 


4.8 


4.4 


4.7 




(1.3) 


(1.4) 




0 


4.1 


4.5 


4.3 




(1.1) 


(1.1) 




RP 


4.1 


4.3 


4.2 




(0.8) 


(1.6) 




Unweighted Means 


5.2 


5.3 


5.2 



*£ < .05. 
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Table 19. C-130 Flight Simulator: Mean Ratings 
(and Standard Deviations) of the Amount 
of Training Received In AIF Use 



Feature 


RTU 


CTU 


Combined 


M 


5.0 
(1.6) 


5.5 

(1.1) 


5.T 




E 


4.9 

(1.4) 


4.8 

(0.6) 


4.9 




CKO 


4.7 

(1.5) 


4.8 

(1.0) 


T.7 




R 


4.6 

(1.2) 


4.8 

(0.8) 


4.T 




T5F 


4.6 

(1.3) 


4.8 

(1.0) 


4.6 




PF 


4.4 

(1.4) 


4.2 

(0.9) 


T.3 




PAM 


4.3 

(1.2) 


4.1 

(1.0) 


4.2 




PRM 


3.6 
(1.5) 


4.2 

(1.0) 


3.T 




AMI 


3.8 
(2.1) 


3.5 
(1.5) 


3.6 




HC 


3.4 

/I ^ \ 

(1.7) 


3.8 
(1.0) 


3.5 




PMS 


2.9 
(1.8) 


3.5 
(1.4) 


3.1 




RP 


2.8 
(1.4) 


3.5 

(1.2) 


3.1 




0 


2.4 

(1.1) 


3.2 
(1.2) 


2.7 


Combined 


3.9 


4.2 


4.0 



25 

Er|c 36 



Table 20. C-130 Flight Simulator: Mean Ratings 
(and Standard Deviations) of the 
Training Value of AIFs 



Feature 


RTU 


CTU 


combined 


M 


6.2 


6.4 


6.2 






In 7\ 
lU./) 




R 


5.5 


6.0 


5.7 






1 1 9> 




TSF 


5.6 


5.6 


5.6 






f 1 ^\ 




E 


5.4 


5.9 


t>.0 










CKO 


5.3 


5.1 








(1 I) 




Pr 




il D 


C 1 










RAM 


4.7 


4.1 


il il 

4.4 










AH I** 


5.2 


3.1 


il 0 






V i .«/ 




PMS 


4.8 


3.7 


4. o 




(1.6) 


(1.8) 




PRM 


4.2 


3.7 


4.0 




(1.7) 


(1.8) 




D 


3.4 


3.4 


3.4 




(1.5) 


(1.8) 




RP 


3.0 


3.4 


3.2 




(1.4) 


(1.4) 




HC 


3.1 


3.2 


3.2 




(1.3) 


(1.6) 




Unweighted Means 


4.7 


4.5 


4.6 



**£< .01. 
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Table 21 . CH-3, HH-53 Flight Simulators: Mean 
Ratings (and Standard Deviations) of 
the Frequency of AIF use 



Fpa^ ijitp 




CTU Combined 


M 


5.3 
(2.4) 


6.6 
(0.9) 


6.T 




E 


4.8 

(1.2) 


6.2 
(1.0) 


5.7 




PAH 


4.9 

(2.2) 


6.0 

(1.0) 


5.6 




TSF 


4.5 

(1.0) 


6.1 
(0.7) 


5.5 




CKO 


5.1 
(2.0) 


• 5.4 
(1.6) 


5.3 




R 


4.3 

(1.3) 


5.5 

(1.1) 


5._1_ 




PF 


4.3 

(1.7) 


4.7 

(1.9) 


_4.6 


RP 


2.2 
(1.2) 


3.8 
(1.4) 


3.2 


0 


1.5 
(1.5) 


1.6 
(1.3) 


1.6 


Combined** 


4.1 


5.1 


4.7 



**p < .01. 



Ease of AIF use . The ease- of- use ratings are summarized In Table 22. 
Except tor demonstration, all the features received high mean ratings. 
There was no overall difference between the RTU and CTU ratings, T « 18.5, 

f> .05, nor were there any significant differences for particular 
eatures. 
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Table 22 . CH-^, HH-53, Flight Simulators: Mean Ratings 

(and Standard Deviations) of the Ease of AIF Use 



heature 


RTU 


CTU 


Combined 


TSF 


6.6 


6.4 


6.5 




(0.5) 


/A C\ 

(O.o) 




M 


6.3 


6.2 


6.3 




/A A \ 

(0.9) 


/I A ) 

(1.0) 




CKO 


6.5 


6.1 


6.2 




/A 7 \ 

(0.7) 


/A D \ 

(0.8) 




PAM 


5.8 


5.5 


5.6 




(1.4) 


(1.3) 




E 


5.3 


5.6 


5.5 






(1 0) 




R 


5.2 


5.6 


5.4 




(0.9) 


(0.8) 




PF 


5.5 


5.1 


5.2 




(1.1) 


(1.6) 




RP 


4.4 


5.2 


5.0 






(1.5) 


(1.0) 


D 


4.5 


3.8 


3.9 




(2.1) 


(1.9) 




Unweighted Means 


5.6 


5.5 


5.5 



Training in AIF Use . The amount of training received in AIF use is 
sunmarlzed in Table 23. Except for demonstration, it appears that the Sis 
received at least moderate amounts of training in the use of each feature. 
The analysis of covariance revealed a significant main effect of AIF, 
£(8,208) ■ 21.98, £ < .001. However, neither the main effect of training 
unit nor the AIF by training unit interaction was significant, F(l,25) « 
2.49, £ > .05 and F(8,208) ■ 1.62, £ > .05, respectively. The Tukey HSD 
test revealed thatT except for record/ pi ay back and demonstration, the 
amount of training received in AIF use was fairly consistent. 
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Table 23. CH-3, HH-53 Flight Simulators: Mean 

Ratings (and Standard Deviations) of the 
Amount of Training Received in AIF Use 





rTU 


CTU uomDined 


M 


4.7 


5.5 


5.T 






(2.2) 


(1.9) 






TSF 


4.5 


5.4 


5.0 






(1.2) 


(1.4) 






E 


3.9 


5.4 


4.8 






(1.6) 


(1.4) 






R 


4.1 


5.0 


4.6 






(1.0) 


(1.6) . 






PF 


3.9 


4.9 


4.5 






(1.4) 


(1.7) 






CKO 


4.1 


4.8 


4.5 






(1.8) 


(1.8) 






PAM 


4.0 


4.8 


4.5 






(1.5) 


(1.7) 






RP 


2.1 


4.5 


3.5 




(0.9) 


(1.2) 




D 


1.5 


1.8 


1.7 




(1.2) 


(1.2) 




Combined 


3.6 


4.7 


4.3 



Training value of AIFs . The training value ratings are summarized in 
Table W. The ratings were very high, with most of the means in the 4.5 to 
6.5 range (i.e., considerable to great ). Only demonstration received a low 
mean rating. There was no significant overall difference between the RTU 
and CTU ratings, T = 7, £ > .05, nor were there any significant RTU/CTU 
differences for particular features. 
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Table 24. CH-3, HH-53 Flight Simulators: Mean 
Ratings (and Standard Deviations) of 
the Training Value of AIFs 



Feature 


RtU 


CTU 


ComDlned 


M 


6.2 


6.4 


6.4 




(1.4) 


(1.1) 




TSF 


6.0 


6.2 


6.1 




(1.2) 


(0.9) 




E 


5.2 


5.8 






(1.5) 


(1.0) 




PAM 


5.5 


5.5 


5.5 




(1.4) 


(1.4) 




R 


5.0 


5.6 


5.t 




(1.3) 


(1.3) 




PF 


5.0 


5.4 


5.2 




(1.4) 


(1.7) 




CKO 


5.1 


4.8 


4.9 




(1.4) 


(1.9) 




RP 


4.1 


5.0 


4.7 




(1.7) 


(1.8) 




0 


3.0 


2.7 


2.8 




(2.3) 


(1.7) 




Unweighted Means 


5.0 


5.3 


5.2 



Potential Training Value of AIFs 

A three-factor (ATD x AIF x Training Unit) repeated measures analysis 
of covarlance (covarlate « number of Instructor-hours) was 'jsed to analyze 
the MAC Sis' potential training value ratines. The analysis r'^'ealed only 
two significant effects: a main effect of AIF, F(15,2265) » : 64, p < 
.001, and an AIF by ATD Interaction, F(30,2265) » 3.00, £ < .001. The 
relevant data are summarized In Table'"25. 
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Table 25. MAC Aircrew Training Devices: 
Mean Ratings of the Potential 
Training Value of AIFs 



Feature C-5A/C-141 C-130 CH-3/HH-53 Combined 



M 


5.7 


6.2 


6.2 


5.9~ 


E 


5.6 


5.9 


6.0 


5^7 


R 


5.6 


5.6 


5.6 


5.6 


TSF** 


5.0 


5.8 


6.2 


5.T" 


PF 


5.2 


5.4 


5.2 




CKO 


4.9 


5.6 


S.l 




AMI 


4.6 


5.3 


5.5 [" 


T9 


PAM* 


4.7 


5.0 


5.7 


4.9 1 


RP** 


4.7 


4.1 


5.4 


— "n 


PRN 


4.8 


4.7 


4.0 


4.6 


PMS 


4.4 


5.0 


4.5 


4.5 


D 


4.5 


4.3 


4.4 [ 




IPT 


4.3 


3.7 


4.1 


4.jy 


RB 


4.1 


4.0 


3.2 




HC 


3.7 


4.0 


4.0 


3.8 


AAT 


3.9 


3.6 


3^ 


3.8 


^ombl ned 


4.7 


4.9 


4.9 


4.8 



*p < .05. 
**p < .01. 



The main effect of AIF was analyzed by the Tukey HSD test, which was 
used to make pal rwlse comparisons among the combined mean ratings. The 
results of that test are shown In Table 25 by the placement of brackets 
around the means that did not differ significantly (p < .01). Thus, the 
features rated highest In potential training value were motion, 
environmental, reset, total system freeze, and paitlal freeze, whereas the 
lowest rated features were demonstration. Instructor pilot tutorial, 
recorded briefing, hard copy, and automated adaptive training. 
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The AIF by ATD Interaction was analyzed by the Dunn test, which was 
used to make palrwlse comparisons among the three mean ratings for each 
feature. Those features for which there was at least one significant 
comparison are marked with asterisks. In each case, the CH-3/HH-53 Sis 
rated that feature significantly higher In potential training value than 
did other Sis. More specifically, for total system freeze and parameters 
monitoring, the CH-3/HH-53 means were significantly hiaher than those of 
the C-5A/C-141, whereas for record/playback, the CH-3/HH-53 mean was 
significantly higher than that of the C-130. 

Interrelations Among the Variables 

Table 26 shows the Intercorrelatlons among the MAC Sis' ratings of 
each feature on each of the five questions. All the coefficients are 
positive and significant, jp < .001, although they are somewhat lower than 
those obtained from the Alt data. Nevertheless, It can be generally stated 
that the more frequently a feature was used, the easier It was to use, the 
more training was received In Its use, and the greater was Its training and 
potential training value. 



Table 26. MAC Aircrew Training Devices: Matrix of 
Intercorrelatlons Among Frequency of Use, 
Ease of Use, Training Received, Training 
Value, and Potential Training Value 



FREQUSE EASEUSE TRECD TVALUE PTVALUE 

FREQUSE 1.00 

EASEUSE .41 1.00 

TRECD .43 .29 1.00 

TVALUE .57 .36 .42 1.00 

PTVALUE .41 .22 .37 .67 1.00 



Note All correlations are significant, £ < .001. 
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Table 27 summarizes the results of a stepwise multiple linear 
regression analysis In which the frequency of AIF use was predicted from a 
wJLS""fl2"y2^?r '•«naJn1no variables. The table Indlcatvs that, 
together, the predictor variables accounted for 41 percent of the variance 
in the frequency-of-use ratings, the most Important predictor being 

f?!I DS.!!^"®*/^^]^*'!?*^" °'"^®'"» 0^ "se and training received. 

(The addition of potential training value did not significantly Increase 
predictability). These results were similar to those obtained from the ATC 
data except that approximately 12 percent less variability was accounted 
for. 



^^^^^ 27. MAC Aircrew Training Devices: Multiple Linear 
Regression of Frequency of AIF Use on Ease of 
Use. Training Received. Training Value, and 
Potential Training Value 



Uepenaent variable: Frequency of Klir Use 

Multiple R: . .64 
Multiple R-Square: .41 
Standard Error of Estimate: 1.52 

Analysis of Variance: 

Sum of Squares OF Mean Square F £ 

Regression 2021.4041 3 673.8013 293734 < .001 

Residual 2937.8682 1279 2.2970 

Summary of Stepwise Regression: 

„ Multiple Increase 

Step Wo. Variable R fej in RSQ 

1 TVALUE .57 .33 .33 

2 EASEUSE .61 .38 .05 

3 TRECD .64 .41 .03 




Strategic Air Command 

FB-lUA Simulator Training Mission 

The FB-111 A mission lasts 3 to 5 hours. Including approximately 1 to 
hours of ground time for prebrleflng and debriefing, and 2 to 3 hours of 
in-flight training. Both peacetime and Emergency War Order (EWO) mission 
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profiles are used. A typical RTU mission consists of various basic skills, 
such as power on/off, preflight, taxi, takeoff, departure, formation, air 
refueling, descent to low level, climb, penetration, bombing, and landing. 
A typical CTU mission includes formation, air refueling, low-level 
navigation aiJ defensive tactics, emergency-procedure recovery to an 
unfamiliar airfield, tank rendezvous, weapons delivery, coramjnication 
procedures, electronic cour-termeasures, and other tactics that comprise the 
EUO mission profile. Selected malfunctions are inserted throughout both 
RTU and CTU missions. 

Frequency of AIF Use 

The frequency- of- use ratings are summarized in Table 28. Most of the 
means are above 4.5 (i.e., frequently ) , which suggests that the frequency 
of AIF use was generally high. The analysis of covariance revealed a 
significant main effect of AIF, F(ll,649) - 129.73, p < .001. and a 
significant AIF by training unlt'interaction, F(ll,649) » 4.01, p < .001. 
The overall difference between the RTU and CTU'ratings was not significant, 
F < .001. 

The Tukey HSD test revealed that parameters and procedures monitoring 
received significantly higher ratings than did all other features, while 
record/playback, demonstration, and hard copy received significantly lower 
ratings than did all other features (£ < .01). The Dunn test ravealed only 
one significant RTU/CTU comparison: programmed -:1siion scenarios were used 
more frequently by CTU Sis than by RIU Sis. 

Ease of AIF Use 

The ease-of-use ratings are sunmarized in Table 29. Ease of use was 
high, with most of the means in the 4.0 to 6.5 range (i.e. moderate to very 
easy ). There was no overall difference between the RTU ana tiu ratings, 
T^26.5, £ > .05, nor were there any significant RTU/CTU differences for 
particular features. 

Training in AIF Use 

The amount of training received in AIF use is sunmarized in Table 30. 
Most of the means ranged from 3.5 to 5.5 (i.e., moderate to considerable ). 
The analysis of covariance revealed a significant mam effect of AIF, 
F(ll,660) « 53.84, £ < .001; however, neither the main effect of Training 
Unit nor the AIF by Training Unit interaction was significant, F(l,59) « 
1.73, £ > .05 and F(ll,660) « 1.79, £ > .05, respectively. 

The results of the Tukey HSD test were similar to those obtained from 
the frequency- of- use data. Parameters monitoring and procedures monitoring 
received significantly higher ratings than all other features, while 
record/playback, demonstration, and hard copy received the lowest ratings. 
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Table 28. FB-lllA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Frequency of AIF Use 



reo uure 


RTU 


CTU Combined 


PAM 


6.8 

(0.5) 


6.6 
(0.8) 


6.7 




PRM 


6.8 
(0.5) 


6.6 
(1.0) 


6.6 




N 


5.8 
(1.8) 


5.8 
(1.5? 


5.8 


E 


5.7 
(1.4) 


5.8 
(1.2) 


5.7 


CKO 


4.9 

(0.9) 


5.5 
(1.3) 


_5T 




PF 


5.4 

(1.1) 


4.6 
(1.6) 


T.8 




PMS** 


3.6 
(2.2) 


5.3 
(1.7) 


4.8 




R 


4.5 
(1.3) 


4.8 

(1.3) 


4.8 




TSF 


4.0 

(1.2) 


4.4 

(1.4) 


_4.3 


RP 


2.5 
(1.2) 


1.7 
(0.8) 


2.T 




D 


1.6 
(0.8) 


1.4 
(0.6) 


1.5 




HC 


1.4 
(0.6) 


1.3 
(0.6) 


l._3. 




Combined 


4.4 


4.5 


4.5 



**p < .01. 



o 
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Table 29. FB-lllA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Ease of AIF Use 



Feature 


RTU 


TTTT 

CTU 


i/OmDinea 


TSF 


6.2 


6.4 


6.3 




(1.0) 


(0.8) 




CKO 


6.3 


6.3 


6.3 




(0.8) 


(0.9) 




PF 


6.1 


6.2 


6.2 




(1.0) 


(1.0) 




M 


5.8 


6.2 


6.1 




(1.3) 


(0.9) 




E 


5.5 


5.8 


5.7 




(0.9) 


(1.0) 




R 


5.4 


5.7 


5.6 




(1.5) 


(1.0) 




PAM 


4.9 


4.5 


4.6 




(1.4) 


(1.4) 




PMS 


4.3 


4.3 


4.3 




(1.6) 


(1.3) 




PRM 


4.8 


4.1 


4 3 




(1.6) 


(1.6) 




RP 


3.8 


3.3 


3.5 




(1.4) 


(1.4) 




D 


2.8 


2.9 


2.9 




(1.5) 


(1.1) 




HC 


2.4 


2.7 


2.6 




(1.4) 


(1.8) 




Unweighted Means 


4.9 


4.9 


4.9 
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Table 30. FB-lllA Operational Flight Trainer: Mean 
Mean Ratings (and Standard Deviations) of 
the Amount of Training Received in AIF Use 



Feature 




CTU Combined 


PRM 


4.9 
(1.4) 


5.6 
(1.2) 


5.5 




PAM 


4.9 
(1.4) 


5.5 
(1.3) 






H 


4.4 

(1.2) 


4.8 

(1.6) 


4.6 


E 


3.9 
(1.2) 


4.8 

(1.2) 


4.5 


CKO 


4.1 
(0.9) 


4.5 

(1.4) 


4.4 




PF 


4.2 

(1.3) 


4.4 

(1.6) 


4.4 




TSF 


4.1 

(1.1) 


4.3 
(1.4) 


4.3 




R 


3.8 
(1.3) 


4.3 
(1.3) 


_i-2 




PMS 


3.2 
(1.6) 


4.0 
(1.8) 


3._7_ 




RP 


2.9 
(1.4) 


2.2 
(1.2) 


T.4 


D 


2.4 
(1.4) 


2.2 

(1.4) ^ 


1,1 


HC 


1.3 
(0.6) 


1.4 

(0.7) 


1.4 


Combined 


3.7 


4.0 


3.9 



37 48 



Training Value of AIFs 

The training value ratings are summarized In Table 31. The ratings 
were very high, with most of the means In the 4.5 to 6.5 '"fnge (I.e., 
considerable to great). Although there were no significant RTU/CTU 
comparisons for parfTcular features, the CTU Sis' ratings were 
significantly higher, overall, than those of the RTU Sis, T . 11, £ < .05. 

Table 31. FB-lllA Operational Flight Trainer: Mean 
Ratings (and Standard Deviations) of the 



Training Value of AIFs 


Feature 


RTU 


ClU 


Comblnea 


PRM 


6.3 


6.4 


6.4 




(1.0) 






PAM 


6.3 


6.3 


6.3 


(1.0) 






PF 


6.2 


5.9 


6.0 


(0.8) 


f 1 o\ 




TSF 


5.5 


5.9 


5.8 




(1.3) 






E 


5.3 


5.6 


5.5 




in Q\ 






N 


5.5 


t). c 


9.0 




(1.2) 


(1.8) 




CKO 


4.9 


5.4 


5.2 




(0.6) 


(1.6) 




R 


4.9 


5.2 


5.1 




(1.5) 


(1.4) 




PMS 


4.3 


5.0 


4.9 


(1.9) 


(1.5) 




RP 


3.9 


3.7 


3.7 


(1.3) 


(1.6) 




0 


3.3 


3.8 


3.7 




(1.6) 


(1.7) 




HC 


2.0 


2.9 


2.7 




(0.8) 


(1.3) 




Unweighted Means* 4.9 


5.1 


5.0 



*p < .05. 



ERIC 



38 

49 



Potential Training Value of AIFs 

The potential training value ratings are summarized in Table 32. Like 
the training value ratings* these ratings were also very high, with most of 

'^^^'^^ 32. FB-lllA Operational Flight Trainer: Mean 
Ratings (and Standard Deviations) of the 
Potential Training Value of AIFs 



Feature 


rYi) 


CTU Combined 


PRM 


fi 7 




6.1 








(0.5) 


(0.51 






PAN 




6.5 


6.6 








(0.5) 


(0.7) 






PF 




5.9 


6.J 








(1.1) 


(1.2) 








TSF 


S ft 




5.9 








(1.1) 


(1.0) 






E 




D.7 


15.9 






(0.8) 


(1.0) 






N 


5 9 


w« 0 


?.7 






(1.2) 


(1.7) 






CKO 


5.5 


R 7 


5.6 






(1.3) 


(1.4) 






R 






5.6 








(1.0) 


(1.1) 






PhfS 






5.4 






(1.5) 


(1.4) 








AMI 




4.9 


T.9 






(2.0) 


(1.5) 






AAT 


4.9 


4.5 


4.6 






(1.8) 


(1.7) 






RP 


5.2 


4.4 


4.6 






(1.3) 


(1.7) 






0 


4.9 


4.4 


4.T 






(1.8) 


(1.5) 






IPT 


4.4 


4.3 


4.3 






(1.7) 


(1.6) 




HC 


4.2 


4.1 


4.1 






(1.9) 


(1.6) 




RB 


3.7 


3.8 


3.2_ 






(1.7) 


(1.6) 






Combined 


5.4 


5.2 


5.3 
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the means In the 4.5 to 6.5 range (I.e., considerable to great ). Even the 
lowest rated feature, recorded briefing , was sin i conslaerea to have 
moderate potential training value. The analysis of covarlance revealed 
only one significant effect, that of AIF, F(15,885) ■ 25.40, £ < .001. 

Interrelations Among the Variables 

Table 33 shows the Intercorrelatlons among the FB-lUA Sis' ratings of 
each feature on each of the five questions. All of the coefficients are 
positive and significant, £ < .001, as they were for the ATC and MAC data. 
Thus, the more frequently a feature was used, the easier It was to use, the 
more training was received In Its use, and the greater Its training and 
potential training value. 



Table 33. FB-lUA Operational Flight Trainer: Matrix 
of Intercorrelatlons Among Frequency of Use, 
Ease of Use, Training Received, Training 
Value, and Potential Training Value 



FREQUSE EASEUSE TRECD TVALUE PTVALUE 



FREQUSE 1.00 

EASEUSE .33 1.00 

TRECD .53 .29 1.00 

TVALUE .58 .32 .53 1.00 

PTVALUE .50 .23 .44 .77 1.00 



Note All correlations are significant, £< .001. 



Table 34 summarizes the results of a stepwise multiple linear regression 
analysis In which the frequency of AIF use was predicted from a linear 
combination of the remaining variables. The table Indicates that, 
together, the predictor variables accounted for 42 percent of the variance 
In the frequency-of-u$e ratings, the most Important predictor being 
training value, followed. In order, by training received, ease of use, and 
potential training value (which contributed relatively little to the 
overall level of predictability). These results were similar to those 
obtained from ATC and from MAC. 
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Table 34. FB-lllA Operational Flight Trainer: Multiple 
Linear Regression of Frequency of AIF Use on 
Ease of Use, Training Received, Training 
Value, and Potential Training Value 



Dependent Variable: Frequency of AIF Use 



Multiple R: .65 
Multiple R-Square: .42 
Standard Error of Estimate: 1.44 



Analysis of Variance: 



Regression 
Residual 



Sum of Squares OF 

1012.0051 4 
1371.5435 660 



Mean Square F 

253.0013 121.75 
2.0781 



Summary of Stepwise Regression: 

Multiple 



Step No. 
1 

. 2 
3 
4 



Variable 

TVALUE 
TRECO 
EASEUSE 
PTVALUE 



T 

.58 
.64 
.65 
.65 



!Die 

.34 
.41 
.42 
.42 



Increase 
In RSQ 

.34 
.07 
.01 
.00 



< .001 



F-to- 
Enter 

337.56 
77.75 
15.04 
5 01 



IV. DISCUSSION 



For purposes of discussion, the 16 AIFs surveyed In Phase II can te 
organized Into four categories. 

Briefing AIFs are designed for briefing the student and SI prior to 
or during a training mission. The purpose Is to establish a learning set 
and to Increase learning readiness. These features Include 

1. Instructor pilot training. 

2. Recorded briefing. 

3. Demonstration. 

Training Management AIFs Include various features designed to control 
the structure and sequencing of tasks within a training mission. These 
features Include 

1. Total system freeze. 

2. Reset. 

3. Automated adaptive training. 

4. Programmed mission scenarios. 
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Variation of Task Diff IcuUv/Fldellty AIFs permit the SI to control 
the difficulty of simulated flight through variations in ATD fidelity, 
configuration, or task load demands. These features include 

1. Automated malfunction insertion. 

2. Partial freeze. 

3. Crash/kill override. 

4. Environmental. 

5. Motion. 

Monitor and Feedback AIFs permit the SI to monitor student 
performance and provide the student with performance feedback. These 
features include 

1. Parameters monitoring. 

2. Procedures monitoring. 

3. Record/playback. 

4. Hard copy. 

Air Training Command 

Brief i no AIFs . None of these features were available on the T-50 and 
T-51 ATDs. Wnile several Sis considered these features to have significant 
potential training value in terms of time and manpower savings, most Sis 
believed that automated briefings would not permit sufficient flexibility. 
For example, during a "dual" mission, it would simply be easier and more 
appropriate for Sis to fly demonstrations themselves. 

Training management AIFs . Total system freeze and reset were among 
the most often used, easily used, and highly valued AIFs, These features, 
which are typically used in succession, permit Sis to temporarily suspend 
the mission in order to offer instruction and then to rapidly re- initialize 
the ATD to a particular configuration. Automated adaptive training and 
programmed mission scenarios were unavailable. Since most Sis preferred to 
manage training themselves, neither feature received particularly high 
potential training value ratings. 

Variation of task difficulty/fidelity AIFs . Environmental and motion 
were among the most frequently used and highly valued features. 
Environmental was especially important for T-37 trainees since it offered 
them their only experience with instrument flying. Partial freeze was used 
significantly more often by T-51 Sis. This did not reflect a difference in 
training value, but rather, was due to a difference in accessibility. 
Partial freeze could only be activated from the T-50 remote console, 
whereas T-51 Sis could freeze altitude, position, and heading at the 
cockpit control panel. 

Automated malfunction insertion was used infrequently by both groups. 
Most instructors preferred to insert malfunctions manually, because this 
was easier and permitted greater training flexibility. Crash override 
received only moderately frequent use, but it was an important feature 
since crashing can cause damage to the terrain model board probe. 
Otherwise crash override was considered to be a convenient means of 
avoiding a reset following a crash. 
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Monitor and feedback AIFs . Of the four monitor feedback features, 
only record/ p I ayoack was available on ^he T-50 and T-51 ATDs. It was used 
with moderate regularity and was rated relatively high In training value. 
Parameters monitoring, procedures monitoring, and hard copy were considered 
to have moderate potential training value; one T-50 SI noted that hard copy 
might be especially useful for Instrument training. 

Differences between T-50 and T-51 Sis' ratings. Although there was a 
significant overall difference between the T-50 and T-51 Sis' frequency-of 
use-ratings, the significant AIF by ATD Interaction suggests that this 
difference was mostly due to the greater use of partial freeze by T-51 Sis. 
It Is not clear, however, why T-51 Sis also reported receiving more 
training in AIF use. 

Military Airlift Command 

Brief ing AIFs . Most MAC Sis felt that Instructor pilot tutorial and 
recorded briefing would not enhance simulator training. "Hands-on" 
training was considered to be superior to Instructor pilot tutorial, and 

face- to- face" J)r1ef ings, which afford opportunity for trainees' questions, 
were preferred over recorded briefings. 

Demonstration, which was available on the C-130 and CH-3 ATDs (it had 
been disabled on the HH-53 ATD), received relatively low ratings. 
Demonstrations were difficult and time-consuming to use, and the available 
examples were not sufficiently representative of the mission profiles. C- 
5A and C-141 Sis felt that demonstrations would probably be unnecessary 
since "transports do not do maneuvers," and crew coordination, which is of 
great importance for a successful mission, cannot be "demonstrated." 

Training management AIFs . On the C-5A and C-141 ATDs, total system 
freeze was generally used only during emergencies or following serious 
errors. At the remaining ATD sites, it was regularly used to correct 
student procedures and to point out errors. Reset was frequently used by 
all MAC Sis. 1 ^ J 

Automated adaptive training was unavailable, and it was rated the 
lowest *■ all features in potential training value. One C-5A SI summed up 
the consensus opinion: "Computers don't know wh^ a student is doing well 
and might Increase complexity before the studeTiT'is ready." Programmed 
mission scenarios were available on the C-130 ATD, but fay were uspj the 
least of all AIFs. Apparently, the available scenarios did not match the 
mission requirements. Most MAC Sis believed that programmed mission 
scenarios could be valuable, but not if training flexibility was 
sacrificed. 

Variation of task difficulty/fidelltv AIFs . Motion and environmental 
were among the most frequently used, easily used, and valuable AIFs. 
Nevertheless, a few C-5A/C-141 and CH-3/HH-53 Sis noted deficiencies in 
reliability and fidelity. Partial freeze was used with moderate regularity 
in various situations. On the C-5A/C-141 ATDs, the Sis frequently used 
position freeze (rather than total system freeze) in order to point out 
student problems. Several C-130 Sis reported using position and altitude 
freeze in order to reduce student task load demands. CH-3 Sis used partial 
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freeze to prevent the simulator from "hitting" obstacles and used it during 
hover checks. 

Automated malfunction insertion was available on the C-130 ATD, where 
it was used moderately often by the RTU Sis but very rarely by the CTU Sis. 
Apparently, the available malfunctions did not conform to the CTU mi??sion 
profile. Although automated malfunction insertion was rated relative^ 
high in potential training value, many Sis expressed the opinion that 
manual insertion was preferable. 

Crash override was frequently used and highly valued at each ATD 
site. On the C-5A/C-141 and CH-3 ATDs, crash override is normally kept in 
the override position in c der to avoid damage to the simulator. C-130 Sis 
used crash override during "non-visual* missions and during stall training. 

Monitor and Feedback AIFs . Parameters and procedures monitoring were 
only occasionally used by t-Uu Sis, many of whom believed it was easier to 
look at tr.e instruments directly. Several CH-3/HH-53 Sis noted that their 
existing monitoring capobility was not sensitive enough to measure the 
rapid changes in parameter values that occur during their mission. Several 
C-5A/C-141 Sis believed that it would be easier to monitor parameters and 
procedures "over- the-shoul der," while others considered these features to 
nave significant potential value since "we miss a lot now." 

Record/playback was among the lowest rated features on the C-130 ATD. 
It was considered time- consuming and relatively difficult to use. On the 
CH3/HH-53 ATDs, r(-:crd/playback was also seldom used, but it was rated much 
higher in training value, probably because it was easier to use. There was 
considerable disagreement among the C-5A/C-i41 Sis as to the potential 
training value of record/pl»yback. 

Hard copy was seldom used by C-130 Sis. Several instructors referred 
to it as "unreliable" and "time-consuming." Hard copy was generally 
considered to have only moderate potential training value; however, a few 
C-5A/C-141 Sis believed it would be useful for ground track recording and 
"accident" investigation. 

Differences between MAC RTU and CTU Sis' ratings . The RTU and CTU 
mission profiles for each ATD were similar. It is not surprising, then, 
that there were relatively few significant differences between the RTU and 
CTU Sis' mean ratings. The only significant overall difference occurred in 
the CH-3/HH-53 data; the CTU Sis reported significantly greater use of 
AIFs than did the RTU Sis. The reason for this difference was not 
immediately apparent, but it is feasible that the greater complexity of the 
CH-3/HH-53 CTU mission required more frequent AIF use. 

Strategic Air Command 

Br iefing AIFs . Instructor pilot tutorial and recorded briefing were 
rated ib having only moderate potential training value. Most FB-lllA Sis 
believed that "hands-on" and "face-to- face" instruction would be 
preferable. The ('emonstration feal ^ was rarely used. It was sometimes 
inoperable, and the available demon^w^ations did not closely conform to the 
FB-lllA mission profile. Nevertheless, some Sis believed that the 
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demonstration feature might be useful for defensive maneuvers and 
instrument training. 



Training management AIFs . The FB-lllA Sis used total system freeze 
and reset with consiaerable regularity, although the level of use was 
somewhat less than it was at most of the other ATD sites. Many FB-lllA Sis 
apparently preferred to use position freeze in order to point out student 
errors. 

Automated adaptive training was unavailable and was rated relatively 
low in potential training value. Programmed mission scenarios, 
particularly the ECM scenarios, were used moderately often. However, many 
Sis commented on the need for manual override. Since the CTU mission 
profile placed greater emphasis on ECM, it seems probable that the 
significant difference between the RTU and CTU Sis' use of progranmed 
mission scenarios reflected this emphasis. 

VaHation of task difficulty/fidelity AIFs . As was the case at the 
other ATD sites, motion and environmental were among the highest rated 
features on each of the five questions. There were several criticisms of 
these features, however. Motion was not always operational, and 
environmental, which was frequently used to vary navigational difficulty, 
was less appropriate for other purposes. 

Partial freeze (i.e., ground position freeze) was frequently used 
instead of total system freeze (i.e., problem freeze) in order to 
temporarily suspend the mission and discuss a problem. Crash/kill override 
was frequently used and was considered to have high training value. T^^e 
FB-lllA Sis, like those at most of the other ATD sites, typically kept this 
feature in the override mode in order to save time and to protect the 
simulator from damage. 

At the time the survey was conducted, automated malfunction insertion 
had only recently been implemented on the SATCOM instructor terminal. 
Automated malfunction insertion was rated as having considerable potential 
training value because it would ease the Si's workload. Nevertheless, like 
most of the Sis surveyed, the FB-lllA Sis generally believed that manual 
insertion would permit greater training flexibility. 

Monitor an d feedback AIFs . Parameters and procedures monitoring were 
'■ated hignest or an features on frequency of use and training value.' The 
comparable ratings were lower from the other ATD sites at which these 
features were available. These differences can be accounted for by 
pointing out that on the FB-lllA ATD, parameters and procedures monitoring 
is done at a remote instructor console. Thus, it is the FB-lllA Si's 
''rl"'^'"^"*^"^ °^ monitoring the simulated aircraft. In contrast, at the 
other ATD sites, the instructor console was located "in the box," which 
permitted Sis to monitor students' performance "over-the-shoulder." 

Both record/ playback and hard copy were available but were rarely 
used. Neither feature was always operational. Moreover, record/playback 
was limited to visual playback only, thus making it unsuitable for student- 
navigator feedback. Hard copy, which was assigned the lowest ratings of 
all available features, was said to yield outpur. that was difficult to 
interpret. 
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Difference s between FB-lllA RTU and CTU Sis' ratings . There were 
very few significant differences between the kiu ana tiu 5ls' mean ratings. 
Of all the available features, only one, programmed mission scenarios, was 
used more frequently by one group (CTU Sis) than by the other (RTU Sis). 
This finding was discussed previously. The only other significant 
difference was in the training value ratings. CTU Sis assigned higher 
training value ratings, overall, than did RTU Sis. The reason for this 
difference was not apparent. In any case, the magnitude of this effect was 
rather small. 

Predicting the Frequency of AIF Use 

The stepwise multiple linear regression analyses of the three sets of 
data (see Tables 11, 27, 34) indicated that at least 40 percent, and as 
much as 53 percent, of the variability in the frequency-of-use ratings 
could be explained by the remaining variables. Training value, alone, 
accounted for over 30 percent in each data set. 

What can be concluded from these facts? Unfortunately, correlational 
findings do not logically imply causality. Instead, they merely reflect 
the likely presence of a relationship between variables. In this case, 
however, it seems reasonable to assume that particular AIFs were used more 
frequently because they had greater training value. Indeed, assuming that 
the training value of an AIF did not affect its use is clearly implausible. 
The remaining variables, i.e., ease of use, amount of training received, 
and potential training value, together accountea lo.- only 10 percent 
additional variability (averaged over the three commands) in the frequency- 
of-use ratings. It seems likely that these variables also influenced the 
frequency of AIF use but to a much lesser extent than did training value. 

Comparisons Between Phase I and Phase II 

The most striking difference between the Phase I and Phase II results 
was in the overall magnitude of the ratings. The ATC, MAC, and SAC Sis 
consistently rated the features higher on all variables than did the TAC 
Sis. This suggests that TAC Sis used AIFs less often, founfJ them more 
difficult to use, received less training in their "se, and considered AIFs 
to be less important for training. As described in Section I of this 
report, these low ratings were due in part to various problems with the 
AIFs, such as hardware and software unreliability, time-consuming 
implementation, functional limitations, and design deficiencies. ATC, MAC, 
and SAC Sis reported these same problems, but their level of AIF use 
remained high. Why was this so? 

One explanation involves the training received by the Sis. The TAC 
Sis reported that they received considerably less training in AIF use than 
did the other MAJCOM Sis. Moreover, the word "informal" was chosen more 
often by TAC Sis when describing the kind of training they did receive. In 
contrast, ATC, MAC, and SAC Sis chose "formal' more often when describing 
their training. !..ese facts suggest that TAC's SI training program is less 
extensive and less structured than are those of the other MAJCOMs. 

There were some similarities in the Phase I and Phase II results. 
The relative ratings of particular AIFs v-sre fairly consistent «»cross all 
ATDs. That is, those features rated highest (lowest) by one group of Sis 
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also tended to be rated highest (lowest) by the other groups. This 
suggests that the overall pattern of AIF use is similar across the MAJCOMs. 
One notable exception was mction. It was consistently among the highest 
rated features at ATC, MAC, and SAC ATD sites; however, it was rated lowest 
of all features in potential training value by the TAC Sis. This 
difference was probably due to the fact that most ATD motion systems are 
not capable of high fidelity simulation of fighter aircraft movement. 

V. RECOMMENDATIONS AND COWCLUSIOW 

At the end of Phase I, it was recommended that certain AIFs need to 
be made more reliable and user friendly before their training effectiveness 
can be ascertained. It was also recommended that a formal intensive 
training program be established in order to teach TAC Sis how to use AIFs 
more effectively. These recommendations apply to Phase II as well, for it 
is clear that most Sis, regardless of command, have not yet fully explored 
the existing instructional capabilities of ATDs. The principles of 
effective AIF use still need to be specified, however. Such principles 
will not be derived from surveys but, rather, from empirical 
Investigations. 
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APPENDIX 

INSTRUCTIONAL FEATURES QUESTIONNAIRE 
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ADVANCED INSTRUCTIONAL FEATURES - IP SlliVEY 

11^^ _Rank ^Squadron Date 

aVING EXPERIENCE: 

Aircraft Total Hours IP Hours 

SIMULATOR EXPERIENCE: 

Siwilator Total Hours IP Hours 

BRIEFLY DESCRIBE A "TYPICAL" TRAINING SESSION GN THIS SIMULATOR: 



GENERAL COMMENTS AND/OR RECOMMENDATIONS: 
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Please familiarize yourself with these instructiorai features and their 
definitions: For each feature, insert 1 (available) or 0 (unavailable; ; 

liM 

Instructor Pilot Tutorial - provides the IP with self -paced programned 

insirucnon in the capabilities and use of the flight Simulator: 

-^PSi " Pe'"'"]ts instructor to "return" the simulated aircraft to a 

stored set of conditions and parameters. 

IkilJij^??/[f^k^ ; instructor to interrupt and suspend 

simulated flight by freezing all system parameters. 

Recorded Briefing - permits instructor to provide student with 

iS?i;";Jl?II„?^?f * structured training session through audio/visual 
■KQio presentation. 

Dwonstration - permits instructor to demonstrate aircraft maneuver (s) 

of ?li;S fliS?.' "'"'"^^^ • standardizeSMS ' 

- permits instructor to record and subsequently 

playback all events that occurred during a segment of simulated flight. 

Environmental - permits instructor to vary environmental conditions 

sjETaTOT^direction and velocity. turbJlence. temperaturS. 

Automated Malfunction Tn<Pr»<«,, . permits instructor to ore-Droaram a 

'*^"5tT component malfunctions and/or emerJeScJ 



Partial Freeze - permits instructor to freeze various flight 
parameters or parameter combinations such as altitude, heading, 
position, attitude, flight systfc.Ti, etc. neauing, 

^1?^/!"'^'^ '^V^ Override - permits instructor to allow simulated 
flight to continue witi^out interruption following a "crash" or "kill." 

2?r"^*Jr**'"V*^^'" *° platform motion system cues such 
as roll, pitch, lateral, vertical, etc. 

Hard Copy - provides a record of alphanumeric and/or graphic 
£br1eff!IS*pJp2ses°" ^^^o^^^ed performance measurement system for 

d?{?7?»l?w^^^lLIr'^"^"3 " computer-controlled variations in task 
difficulty, complexity, and sequence based on pilot's performance! 

Prograiwed Mission Scenarios . computer -control led standardized 
training sessions based on pre-prograamed event sequences. 

'Permits instructor to monitor discrete actions 
EhS^kim. accordance with a procedurally defined 

Parameters Monitor i no - permits instructor to monitor various 
n!l!fl!l!??"* settings, aircraft states, or 
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navigational profiles 
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SUMMARY 



Aircrew training devices (ATDs) are often equipped with sophisticated 
hardware and soft**'are capabilities that permit a simulator instructor (SI) 
to control, monitor, record, and fabricate flight simulation training 
missions. These advanced instructional features (AIFs) reflect the primary 
role of the AID as a flight trainer. The training value of an ATD is a 
function of the degree to which it simulates a particular aircraft and the 
way in which it is used as an instructional device. 

AIFs are costly to implement and in order to justify these costs, 
several questions must be answered. How frequently are AIFs used? How 
easy are they to use? Are simulator instructors adequately trained to use 
AIFs? Do AIFs have sigr.ificant training value? 

This report describes the third phase of a three-phase project 
designed to obtain answers to these questions by surveying simulator 
instructors from the Air Force Major Commands. An on-site survey was 
administered to 159 Sis assigned to replacement training units and 
continuation training units at principal Air Training Command (T-5), 
Strategic Air Command (T-4, B-52 Weapon System Trainer, FB-lllA), and 
Tactical Air Command {F-4G, A-10) ATD facilities. The survey requested 
background information, along with five seven-point rating scales for 
evaluating each of 14 AIFs. Written comments concerning the 14 AIFs or the 
ATD were solicited. 

Based on the utility and utilization ratings, the T-5 and T-4 
trainers were the most favorably evaluated devices surveyed. They were 
followed, in order, by the F-4G simulator, B-52 WST, and A-10 simulator. 
Mission control features (e.g., freeze, reset, and programmed and manual 
threat control) were generally rated high in utility and utilization, 
whereas briefing features (e.g., instructor tutorial, recorded briefing, 
demonstration) and feedback features (e.g., hard copy, record/playback, 
electronic warfare performance scoring) tended to receive lower ratings. 

The level of AIF use was affected somewhat by hardware and/or 
software unreliability, implementation time, functional limitations, and 
design deficiencies. The perceived training value of a feature was the 
most important determiner of its use. 
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PREFACE 



This project was conducted to satisfy requirements of Air Force Human 
Resources Laboratory Technical Planning Objective 3, the thrust of which is 
aircrew training effectiveness. The general objective of this thrust is to 
identify and demonstrate cost-effective simulator training strategies and 
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AIRCREW TRAINING DEVICES: UTILITY AND UTILIZATION 
OF ADVANCED INSTRUCTIONAL FEATURES 
(PHASE III - ELECTRONIC WARFARE TRAINERS) 



I . INTRODUCTION 

An Aircrew Training Device (ATD) serves two functions. First, it is a 
ground-based substitute aircraft that permits student flight crews to fly 
in a safe and carefully controlled environment. More importantly, an AVO 
is, as its name implies, a teaching machine that is designed to facilitate 
the acquisition of flight crew skills. I;i order to fulfill this second 
function, an ATD is equipped with sophi»;ticated hardware and software 
capabilities that permit a simulator instructor (SI) to control, monitor, 
and fabricate simulator ti fining missions. These capabilities, which are 
listed in Table 1, are known as advanced instructional features (AIFs). 
The list was compiled from several sources, but it was drawn primarily from 
Semple, Cotton, and Sullivan's (1981) extensive report describing the AIF 
capabilities of various military and commercial devices. 



Table 1. Advanced Instructional Features 



BRIEFING FEATURES 



Recorded Briefing permits simulator instructor to provide a student with 
information about the simulator and/or a training mission through 

audiovisual media presentation.^ 



Demonstration permits simulator nstructor to demonstrate optimal aircrew 
performance by means of prerecording and subsequently playing back 

segments of simulated flight. ^ 



Instructor Tutorial provides simulator instructor with self-paced 
programmed instruction in the capabilities and use of the simulator. 
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Table!. (Continued) 



TRAINING MANAGEMENT FEATURES 

Total System Freeze permits simulator instructor to suspend simulated 
flight by freezing all system parameters.^ 

Reset permits simulator instructor to return the simulated aircraft to a 
stored set of conditions and parameters.^ 

Crash and/or Kill Override permits simulator instructor to allow 
simulated flight to continue without interruption following a "crash" or 
"kill." 

Automated Adaptive Training is the computer-controlled variation in 
task difficulty, complexity, and/or sequence based on student's 
performance. 

Programmed Mission Scenarios are computer-controlled standardized 
training missions based on preprogrammed event sequences.^ 

Manual Mission Control permits simulator instructor to modify programmed 
scenarios during a training session.^ 

VARIATION OF TASK DIFFICULTY FEATURES 

Automated Malfunction Insertion permits simulator instructor to 
preprogram a sequence o* aircraft component malfunctions and/or 

emergency conditions.^ 

Manual Malfunction Insertion permits simulator instructor to modify 
preprogrammed malfunctions dtring a training session. 
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Ta ble 1. (Continued) 



Environmental permits simulator Instructor to vary environmental 
conditions such as wind direction and velocity, turbulence, temperature, 
and visibility. 

Dynamics permits simulator instructor to vary flight dynamics 
characteristics, such as stability, system gain, cross-coupling, etc. 

Motion permits simulator Instructor to provide a student with platform 
motio'; system cues such as roll, pitch, and yaw. 

Flight System Freeze permits simulator Instructor to simultaneously 
freeze flight control and propulsion systems, position, altitude, 
and heading. 

Position Freeze permits simulator Instructor to simultaneously freeze 
latitude and longitude. 

Attitude Freeze permits simulator Instructor to simultaneously freeze 
pitch, bank, and heading. 

Parameter Freeze permits simulator Instructor to freeze any one or a 
combination of flight parameters.' 

MONITORING FEATURES 

Closed Circuit TV permits simulator Instructor to monitor student's 
behavior from the instructor console. 

Repeaters/Annunc iators provide simulator Instructor with replicas or 
an-ilog representations of flight instruments and controls at the 
instructor console. 
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Table 1. (Concluded) 



Instructor Console Displays permit simulator instructor to monitor 
parameters and procedures at the instructor console by means of 

alphanumeric and/or graphic CRT displays of performance data.^ 

Automated Performance Alert provides simulator instructor with visual 
and/or auditory signals that indicate specific performance deficiencies. 

FEEDBACK FEATURES 

Record/Playback permits simulator instructor to record and subsequently 
play back a segment of simulated flight." 

Automated Performance Fgprihack provides a student with visual and/or 
auditory signals (including verbal messages) that identify performance 
deficiencies. 

Automated Voice Controller is the computer-based technology that 
simulates the role of a controller by combining speech generation, 
speech recognition, and situation awareness capabilities. 

Hard Copy provides a record of alphanumeric and/or graphic 
performance data.* 

Pe rformance Scoring provides a metric that summarizes aircrew task 
performance during a simulated mission. * 



* These features were included in the Phase III questionnaire. 



It appears that military ATDs are more often treated as substitute 
aircraiFt than as teaching machines. A recent report by the United States 
General Accounting Office (1983) concluded that the Armed Services have not 
sufficiently analyzed their training requirements for simulators. Nor have 
they adequately incorporated simulators into their training programs. In 
justifying the purchase of ATDs, the Services have focused instead on 
^duplicating the actual weapon systems and their surroundings. ..with little 
re/erence to how the devices could meet training needs" (p.4). 

By providing AIF capability, simulator manufacturers apparently 
recognize that the training value of an ATD is determined not only by the 
degree to which it faithfully mimics a particular aircraft, but also by the 
way that it is used (Caro, 1973). Previous research suggests that 
effective AIF-based simulator training is practicable (see Polzella, 1983, 
p.8). However, instructional features are expensive to implement, 
especially those features that require the development of complex software. 
In order to justify the^? costs, some questions concerning the present and 
potential utility and utilization of AIFs should be answered: How 
frequently and easily are AIFs used? Are simulator instructors adequately 
trained to use AIFs? Do AIFs have significant training value? 

The present investigation was conducted at the request of the 
Simulator System Program Office (SimSPO) of the Air Force Systems Command, 
Aeronautical Systems Division (AFSC/ASD) in order to answer these 
questions. The specific objectives of this investigation were: 

1. To document and compare the utilization (i.e., frequency and ease 
of use) of AIFs. 

2. To document and compare the utility (i.e., training value) of 

AIFs. 

3. To compare the utility and utilization patterns of AIFs in 
replacement (e.g., basic, primary, lead-in, initial, transition) and 
continuation (e.g., advanced, follow-on, refresher, operational) training 
units. 

A broader objective of this investigation was to provide a database that 
could be helpful both in defining the requirements for ATD procurements and 
in developing future ATD training programs. 

These objectives were to be accomplished in three phases by means of a 
survey of simulator instructors from the Air Force Major Commands 
(MAJCOMs). Phases I and II have already been completed, and the results of 
those surveys are documented in two earlier reports (Polzella, 1983, 1985). 

The subjects in Phase I were 134 simulator-qualified Instructor Pilots 
and Weapons Director Instructors (WDIs) assigned to Replacement Training 
Units (RTUs) and Continuation Training Units (CTUs) at F-4E, F-4G, F-15, A- 
10, and E-3A Tactical Air Command (TAC) training sites. The results 
indicated that most TAC Sis received little training in AIF use and that 
most features were not used very often. Several factors appeared to have 
contributed to the low usages: (a) hardware and/or software unreliability, 
(b) time-consuming implementation, (c) functional limitations, and (d) 
design deficiencies. The results of a multiple regression analysis 
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Indicated that ea$e of use and training value accounted for most of the 
varloblllty In the frequency-of-use ratings. 

The utility and utilization of particular AIFs dif fired both as a 
function of ATD and of training unit. For example, features such as freeze 
and reset were generally used more often during RTU missions, whereas 
programmed mls.'^lon scenarios were generally used more often during C^ii 
missions. These dif ""erences appeared to reflect differences In the 
respective training missions. Thus, RTU missions characteristically 
Include a series of discrete procedural exercises, whereas lengthier 
scenarios are common during CTU missions. 

The subjects In Phase II were 273 s1mulator*qual1f led Instructor 
pilots (IPs), Instructor flight engineers (IFEs), and Instructor radar 
navigators (IRNs) assigned to Air Training Command (ATC; T.37, T-38), 
Military Airlift Command (MAC; C-5A, C-141, C-130, CH-3, HH-53) or 
Strategic Air Command (SAC; FB-lUA) ATD training sites. 

The ;»ost striking difference between the Phase I and Phase II results 
was In the overall magnitude of the ratings. In comparison to the TAC Sis, 
the ATC, MAC, and SAC Sis used AIFs more often, found them easier to use, 
received more training In their use, and considered AIFs to be more 
Important for training. The results suggested that TAC's training program 
for Sis Is less extensive and less structured than those of the other 
MAJCOMs. 

The level of AIF use among ATC, MAC, and SAC Sis was affected somewhat 
by hardware and/or software unreliability. Implementation time, functional 
limitations, and design deficiencies. However, training value appeared to 
be the most Important determiner of AIF use. 

Based on the results of Phases I and II, It was recommended that 
future procurement of AIFs be preceded by a detailed front end analysis 
that clearly relates AIF capability to training needs. The analysis should 
r^'islder all known training applications of the simulator as well as any 
major constraints In the operational environment. During procurement, AIF 
specifications should be prepared to meet user needs and to ensure 
equipment reliability. After operational deployment, the user should 
provide adequate Instructor/operator training in AIF use. 

Phase III, which is described in this report, extended the survey to 
electronic warfare Instructors from ATC, TAC, and SAC. 



II. METHOD 
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Subjects 

The subjects in Phase III were 155 simulator-qualified electronic 
warfare instructors (lEU), weapon systems officers (WSOs), aerial gunnery 
instructors (lAGs), and radar navigator instructors (IRNs). The 
distribution of Sis among the various ATD sites surveyed is shown in Table 
2. Also included in that table are the Sis' mean (and standard deviation) 
number of hours of instructor experience. 
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Table 2. Simulator Instructor (Sis) Surveyed in Phase III 



Command ATD 


ATD-SITE 


Type of 
training 


Type of 
SI 


N 


Instructor 
hours 


ATC T-5 


Mather 
AFB 


Basic 


lEW 


19 


287.4 
(276.0) 


SAC T.4 (B-52) 


Castle 
AFB 


Transition 


lEW 


20 


731.2 
(754.6) 




Mather 
AFB 


Operational 


lEW 


8 


188.9 
(224.6) 


WST (b'bz) 


Castle 
AFB 


Transition 


lEW.IAG 


9 


674.9 
(747.8) 




Wurtsmi th 
AFB 


Operational 


lEW.IAG 


6 


396.2 
(231.9) 


FB-IIIA^ 


Plattsburgh 
AFB 


Tranoition 


IRN 


11 


677.3 
(426.2) 




Plattsburgh 
AFB 


Operational 

< 


IRN 


9 


175.6 
(213.8) 




Pease 
AFB 


Operational 


IRN 


12 


210.2 
(18?. 7) 


TAC F-4G 


George 
AFB 


Replacement 


lEW 


13 


128.4 
(116.1) 




George 
AFB 


Continuation 


lEW.IP 


19 


73.1 
(56.4) 


A- 10 


Davis- 
Monthan AFB 


Replacement 


IP 


16 


98.8 
(85.8) 




England 
AFB 


Continuation 


IP 


17 


52.6 
(37.4) 



T55 



Data from these sites were collected during Phase II. 
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Questionnaire 



The questionnaire that was used to survey the Instructors is shown in 
Appendix A, A^hough it is similar to those used during Phases I and II 
(see Polzella, 1983, Appendix A; 1985, Appendix), several important 
modifications were incorporated- 

The first page of the questionnaire requested information concerning 
flying and simulator experience, the type of training in ATD operations 
received by the Sis, a description of a typical simulator training session, 
and general comments and/or recommendations. 

The second page of the questionnaire included a list of 14 AIFs (drawn 
from the list in Table 1) and their definitions, and a space next to each 
feature that was used to indicate the operational status of that feature 
(e.g., no such capability, and capability present but unreliable, and 
capability present and reliable). 

On subsequent pages were five questions concerning the utility and 
utilization of each feature. 

1. How often have you used it? 

2. How difficult/easy is it to use? 

3. How inadequate/adequate was the training you received in its use? 

4. As presently implemented, how useful is it? 

5. How potentially useful is it? 



For the fifth question. Sis were to assume that they had no prior 
knowledge of the features and to base their responses on the feature 
definitions alone. This question was included in order to achieve a common 
basis for comparis^'-n among all Sis. This w^s not otherwise possible 
because the various ATDs were not similarly equipped. 

Responses to each question were indicated by checking the appropriate 
interval along a seven-point, successive-category rating scale. (On 
certain questions a 0-interval was included for indicating "not 
.applicable. The intervals of each scale were labeled with descriptive 
adjectives in order to facilitate responding and to help interpret the 
ratings. Additional space was provided for comments. 

Procedure 



The questionnaire was administered on-site to various sized (N « 5 to 
10) groups of Sis. The Sis were briefed on the purpose of the 
investigation and copies of the questionnaire were distributed and 
thoroughly reviewed prior to being filled out. For the most part, the 
questionhaire was s^lf-explanatory. However, Question 3 (How 
inadequate/adequate vas the training you received?) required some 
additional instruction. For this questlcn, the Sis were asked to rate each 
feature twice. The fi^st rating assessed the training received in the 

o ' fi^ BEST COPY AVAILABLE 
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operation of that feature, whereas the second rating assessed the training 
received in the effective use of that feature (in terms of student 
outcome). 

The questionnaire could be completed in approximately 30 minutes. 



III. RESULTS 



Table 3 lists the 14 AIFs that were included in the questionnaire 
along with their definitions. The table also lists a mnemonic code for 
each feature, which will be used in subsequent tables. The AIF 
capabilities of the various ATDs are shown in Table 4. 

The Phase III questionnaire yielded the following data from each SI: 

1. Number of hours of flying and simulator experience. 

2. Type of training received. 

3. Description of a typical training session. 

4. Assessment of the operational status of each AIF. 

5. Ratings of the AIFs on each of the five questions (including the 
two ratings of Question 3). 

6. Comments. 

The data were classified by ATD (T-5, F.4G, A-10, T.4 WST, FB-lllA), level 
of training (e.g., transition, operational). Question (1 through 5), and 
AIF (1 through 14). The resulting data matrix was unbalanced due to 
differences in the numbers of Sis and in the AIF capabilities of the 
various ATDs (see Tables 2 and 4). In most cases, this necessitated 
analyzing the data from each ATD separately. 

Descriptive statistics were computed for type of training received by 
the Sis and for their assessment of the operational status of each AIF. 
Multivariate analyses of variance were used to analyze the ratings of each 
feature across the first four questions. The data from Question 5 were 
analyzed separately. The multivariate model was bosed on a two-factor 
mixed design in which level of li fining was the between-subjects factor, 
AIF was the within-subjects factor, and the ratings on the first fo»jr 
questions were the dependent variables. Missing data were deleted "list- 
wise," i.e., subjects who did not rate a feature on all four questions were 
eliminated from the analyses. 

Approximate F-values, derived from Wilks' lambda, were used to test 
the overall multivariate significance of each effect in the model, i.e , 
AIF, level-of-training, and the AIF by level-of-training interaction, w* 'le 
univariate Fs were used to test the significance of these effects for each 
of the four"'questions separately. Tukey honestly significant difference 
(HSD) values were computed for each univariate analysis. These values were 
used to determine significant differences between particular ratings, e.g.. 
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rable 3> Advanced Instructional Features Included in the 
Phase III Questionnaire 



Code Feature 

IT Instructor Tutorial - provides the instr ctor with self-paced 

programmed Instruction in the capabilities and use of the simulator. 

R Reset - permits instructor to "return" the simulated aircraft to a 
stored set of conditions and parameters, 

TSF Total System Freeze - permits instructor to interrupt and suspend 
simulated flight by freezing all system parameters* 

PF Partial Freeze - permits instructor to freeze various flight 

parameter combinations such as altitude, heading, position, attitude, 
flight system, etc, 

RB Recorded Briefing - permits instructor to provide student with 

information about a structured training session through audio/visual 
media presentation. 

D Demonstration - permits instructor to demonstrate optimal electronic 
warrare procedures by prerecording and subsequently playing back a 
simulated engagement. 

RP Record /PI ay back - permits instructor to record and subsequently 
playback a segment of simulated flight. 

AMI Automated Malfunction Insertion - permits instructor to preprogram a 
sequence of aircraft component malfunctions and/or emergency 
conditions, 

HC Hard Copy - provides a record of alphanumer-^c and/or graphic 
performance data for debriefing purposes. 

PTC Programmed Threat Control - computer-controlled standardized 
training sessions based on preprogrammed event sequences. 

MTC Manual Threat Control - permits instructor to modify threat scenarios 
during a training session. 

PRM Procedures Monitoring - permits instructor to monitor discrete 

actions performed by the student in accordance with a procedurally 
defined checklist. 

PAM Parameters Monitoring - permits instructor to monitor various 
instrument readings, control settings, aircraft states, or 
navigational profiles. 

EWS Electronic Warfare Performance Scoring - proviJes a performance 
metric that summarizes the outcomes or EU engagements. 
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Table 4. AIF Capability of Each ATD 



Feature T-5 T-4 WST FB-lllA^ F-4G A-10 



IT 














R 


X 


X 


X 


X 


X 


X 


TSF 


X 


X 


X 


X 


X 


X 


PF 






X 


X 


X 


X 


RB 


X 


X 










D 












X 


RP 






X 






X 


AHI 


X 


X 


X 




X 


X 


HC 


X 




X 


X 


X 




PTC 


X 


X 


X 


X 


X 


X 


MTC 


X 


X 


X 


(X) 


X 


X 


PRM 


X 


X 


X 


X 




X 


PAN 


X 


X 


X 


X 




X 


EWS 


X 




X 


(X) 




X 



^FB-lllA data were collected during Phase II using a different 
version of the questionnaire. Data are not available for those features 
that are In parentheses. Programmed Threat Control (PTC) was listed as 
Programmed Mission Scenarios (RMS) on the Phase II questionnaire. 
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training value of manual threat control vs. training value of programmed 
threat control, frequency of RTU use of reset vs. frequency of CTU use of 
reset. 

The interrelations among the utility and utilization ratings were 
determined by means of correlation an^ regression analyses. First, 
intercorrelations were computed between the ratings of each feature across 
the five questions. Second, multiple linear regression analyses were used 
to determine those variables which significantly predicted the frequency of 
AIF use. Three potential predictors were evaluated: the ease of AIF use, 
the adequacy of training received (a composite variable representing the 
average of the two ratings on Question 3), and AIF usefulness (a composite 
variable representing the average of the ratings on Questions 4 and 5). 
Missing data were deleted "list-wise" from these analyses. 



Air Training Command 



T-5 Trainer 

Training mission . The T-5 ATD is a sophisticated generic trainer for 
primary level electronic warfare skills. The typical training session 
lasts 3 to 4 hours and includes a 15-minute prebriefing of mission 
objectives, a 2 1/2-to 3 1/2-hour mission, and a 30-minute 
debriefing/critique. A complete mission, from takeoff to landing, normally 
requires the student to search for, identify, and determine the parameters 
of electronic warfare (EW) signals and select appropriate countermeasures. 
The instructor's role is to (a) monitor student progress for speed and 
accuracy, (b) freeze and offer feedback verbally and through demonstration, 
and (c) reset as required. 

Training of Sis . On the average, formal classroom instruction 
accounted tor z&% of the initial training for the T-5 Sis (SD = 29%). Only 
11% of the Sis reported having received any refresher training, and nearly 
all of that training was characterized as informal. 

AIFs. The operational status of each AIF on the T-5 trainer is 
summarized In Table 5. A small percentage of Sis indicated that there are 
operational problems with recorded briefing and programmed threat control. 
Otherwise, all availab^ features eppear to operate reliably. The table 
show« that a substant.«.< oportion of Sis apparently have never operated 
auto inalfunction insertion (0.32), manual threat control (0.58), or 
electronic warfare performance scoring (0.28). (Note: These proportions 
include those Sis who indicated "no such capability" for these features.) 

Utilization and utility ratings . The ratings for the T-5 Sis are 
summarized in Table 6. Means and standard deviations are listed for 
available features under Q* estions 1 to 4 and for all features rnder 
Question 5 (potential usefulness). The multivariate analysis or variance 
revealed a significant overall effect of AIF, F(45,463.85) « 4.25, p < 
.001. Each univariate F (df = 9,107) was also significant, £ < .001. 
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Table 5> T-5 Trainer: The Number of lEWs Indicating 
the Operational Status of Each AIF 



No such Never 
Feature capability operated Unreliable Reliable 



IT 


16 


3 


0 


0 


R 


0 


0 


1 


18 


TSF 


0 


0 


0 


19 


nr 
PF 


12 


2 


0 


4 


RB 


2 


1 


3 


13 


D 


15 


2 


1 


1 


RP 


18 


1 


0 


0 


AMI 


2 


4 


0 


13 


HC 


1 


2 


0 


16 


PTC 


0 


1 


2 


16 


MTC 


2 


9 


0 


8 


PRM 


0 


1 


0 


18 


PAM 


3 


1 


0 


15 


EWS 


0 


4 


0 


15 



ERIC 
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Table 6 , T-5 Trainer: Mean Ratings (and Standard Deviations) 
of the Frequency of Use, Ease of Use, Adequacy of 
Training Received, Training Value, and Potential 
Training Value 



Feature 






TRECD(2) 


TRECO (2) 


TVALUE 


PTVALUE 


IT 












4.2 














(1.9) 


R 


4 5 


5 9 




D . 1 


f\ 4 


U • U 




(1.5) 


(0.5) 


(1.6) 


(1.0) 


(0.8) 


(0.6) 


TSF 


5.3 


6 2 


6 7 


6 6 


6 J 


6 5 




{2!l) 


(0.4) 


(0.5) 


(0.5) 


(1.3) 


(0.8) 


PF 












4 6 














(2.2) 


RB 


4.2 


5.8 


6.4 


6 2 


5.6 


4 9 




(2.1) 


(1.2) 


(1.0) 


(1.0) 


(i.'a) 


(1.8) 


D 


























(1.4) 


RP 


























(1.4) 


AMI 




J m <J 


^ m C 










(1.6) 


(1.7) 


(2.5) 


(1.9) 


(1.6) 


(1.7) 










f\ 1 
U • 1 




u • O 




(1.7) 


(1.0) 


(1.1) 


(1.3) 


(1.7) 


(1.0) 


PTC 


6 4 


6 2 


5 9 


1 

O • 1 




f\ A 




(1.1) 


(1.1) 


(1.7) 


(1.6) 


(0.7) 


(0.9) 


MTC 


1.6 


3.9 


2.7 


3.d 


2.9 


4.5 




(0.8) 


(1.1) 


(1.4) 


(1.3) 


(1.6) 


(1.6) 


PRM 


6.5 


6.1 


6.5 


6.3 


6.1 


6.5 




(1.4) 


(0.8) 


(0.9) 


(1.2J 


(1.8) 


(1.0) 


DAM 


0, U 




0.4 


5.9 


D.O 


6.0 




(1.9) 


(0.8) 


(0.9) 


(1.5) 


(2.2) 


(1.5) 


EWS 


5.3 


6.4 


6.5 


6.6 


6.2 


6.3 




(2.1) 


(1.0) 


(0.8) 


(0.6) 


(1.5) 


(1.2) 




4.6 


5.8 


6.0 


6.0 


5.7 


5.6 




(2.2) 


(1.2) 


(1.5) 


(1.4) 


(1.8) 


(1.6) 



Frequency of use was highest for programmed threat control, procedures 
monitoring, and parameters monitoring. These features were used at an 
average rate of at least five times each mission. Frequency of use was 
lowest for auto malfunction Insertion, hard copy, and manual threat 
control. These features were used significantly less often than most of 
the AIFs (Tukey HSD q^* 2.08). With the exception of manual threat 

control, ease of use was uniformly high for all AIFs (Tukey HSD qj « 

1.13). The training received by the Sis was apparently adequate despite 
the lack of both formal classroom Instruction and refresher training. (See 
previous section.) This was not the case for manual threat control, 
however. Training In Its operation and In Its effective use was not judged 
to be adequate. Manual threat control received significantly lower ratings 
on these variables than did all other AIFs (p < .05). A similar pattern 
emerged for the training valje ratings. Most of the features were judged 
to be at least very useful, whereas manual threat control was rated the 
least useful of all AIFs (Tukey HSD qj « 1.51). 

The separate analysis of the potential training value ratings 
(Question 5) also yielded a significant effect of AIF, F(13,231) « 8.65, £ 
< .001. Every feature was judged to have at least moderate potential 
usefulness (Including manual threat control); however, those features that 
are presently unavailable (I.e., Instructor tutorial, partial freeze, etc.) 
were generally rated lower than were those AIFs that are already 
Implemented (Tukey HSD Qg « 1.42). 

Interrelations among utilization and uti l ity ratings . Table 7 shows 
the Intercorreiatlons among the ratings of each feature on fiach of the five 
questions. All the coefficients were positive and significant, £< .01. 
Thus, a feature's rating on any question can be predicted with greater than 
chance accuracy given Its rating on any other question. For example, the 
more useful a feature was, the more frequently It was used, the easier It 
was to use, tie more adequate was the training In Its use, and the greater 
was Its potential training value. However, these predictions would not be 
equally precise. The coefficients of determination (I.e., the squared r 
values) ranged from .07 (FREQUSE/TRECD(2)) to .64 (TRECD(1)/TRECD(2)) over 
the entire atrlx. 

Table 3 summarizes the results of a multiple linear regression analysis 
In which the frequency of AIF use was predicted from a linear combination of 
ease of use, adequacy of training received (a composite variable representing 
the average of the two ratings on Question 3), and training value (a 
composite variable representing the average usefulness and potential 
usefulness). The table Indicates that, together, the predictor variables 
accounted for approximately 30X of the variability In the frequency-of-use 
ratings. However, the only significant predictor was training value. 

Comments . Most of the comments concerning the T-5 ATO were favorable. 
However, several Instructors noted that Instructor training end programming 
support are Inadequate; consequently, many of the advanced capabilities of 
the T-5, such as computer-aided Instruction, are not fully u.illzed. A 
significant operational deficiency Is that the T-5 Is too slow to adequately 
monitor rapidly performed procedural checklists. 
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Table 7. T-5 Trainer: Matrix of Intercorrelations Among 

Frequency of Use, Ease of Use, Adequacy of Training 
Received, Training Value, and Potential Training Value 





FREOUSE 


EASEUSE 


TRECDd 1 




PTVAI IIF 


FREQUSE 


1.00 










EASEUSE 


.31 


1.00 








TRECD(l) 


.37 


.68 


1.00 






TRECD(2) 


.26 


.62 


.80 


1.00 




TVALUE 


.54 


.37 


.48 


.55 1.00 




PTVALUE 


.41 


.26 


.32 


.34 ,62 


1.00 




Note 


All correlations are 


significant, £ < .01. 





Table 8. T-5 Trainer: Multiple Linear P jression of 
f-requency of Use o,i Ease of Use, Adequacy of 

Training Received, aid Training Value 

DEPENDENT VARIABLE: Frequency of AI. Use 

MULTIPLE R: .55 STD. ERROR OF EST. : 1.56 

MULTIPLE R-SQUARE: .30 
ANALYSIS OF VARIANCE: 



REGRESSION 


Sum of Squares df 
144.2528 -J 


Mean Squares 


F-Ratio 
19.844 


ToSo 


RESIDUAL 


329.5401 


136 


2.431 






Predictor 
variable 


Coefficient 


Standard 
error 


Standard 

regression 

coefficient 


t 




EASEUSE 


.3095 


.1943 


.1564 


1.593 


.1135 


TRECD 


-.0314 


.1437 


-.0232 


-.218 


.827?) 


TVALUE 


.9342 


.1574 


.4914 


5.935 


.0000 


(CONSTANT) 


-2.2552 


1.0801 
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^tC^li9.i?.^^iC Com mand 



T-4 T raiOier 

Trajjiing mission. The T-4 ATD provides both transition and 
operafTonaTTrafnfng for B-52 electronic warfare officers. The typical 
training session lasts 2 to 3 hours and includes a 15-minute prebrief ing of 
mission objectives and interference checks, a 2-hour mission, and a 15- 
minute debriefing/critique. The transition and operational missions are 
highly similar. Bot'i missions require the student to recognize and 
counteract a series of ^i^lreats encountered during high-level, low-level, 
and over-water penetrations nto enemy territory. Various malfunctions and 
emergencies are distributed throughout the mission. Although a mission can 
proceed under computer control, the T-4 SI can modify mission flow and 
content through the discretionary use of manual insertions, freezes, and 
resets. 



Training of Sis . The T-4 Sis received both formal and informal 
instrucETon. Formal classroom instruction accounted for 38% of T-4 RTU 
Sis' initial training (SD = 31%), and 70% of T-4 CTU Sis* Initial training 
(SD = 25%). Also, 45% of the RTU Sis and 38% of the CTU Sis reported 
having received refresher training (approximately once within the preceding 
year). However, 26% of the RTU refresher trainina and 80% of the CTU 
refresher training were characterized as formal classroom instruction. 

AIFs. The operational status of each AIF on the T-4 trainer is 
summarized in Table 9. All available features, except recorded briefing, 
appear to operate reliably. 

Utilization and utility ratings. The ratings by the T-4 RTU and CTU 
Sis afe^sunimarTzed^Tn Tables 10 and 11, respectively. Means and standard 
deviations are listed for available features under Questions 1 to 4 and for 
all features under Question 5 (potential usefulness). The multivariate 
analysis of variance revealed a significant overall effect of AIF, 
F(25,410.13) = 4.34, £ < .001. Except for training value, each univariate 
F (df s 5,114) was also significant, £ < .001. Neither level of training, 
nor the AIF by level of training Interaction was significant at the 
multivariate level. 



Frequency of AIF use was high overall. The average feature was used 
at least two to four times a mission. Procedures and parameters monitoring 
were used most often, threat control (programmed and manual) and freeze, 
slightly less so. Recorded briefing was used hardly at all (Tukey HSD = 

.88). Ease of use was also high overall, although the means were not 
statistically equivalent (Tukey HSD qj « .84). The training received by 

the Sis (In both the operation and effective use of the features) was 
judged to be "very adequate." Moreover, each feature (except recorded 
briefing, as rated by the transition Sis) was judged to be "very useful." 

The aparate analysis of the potential training value ratings revealed 
a significant effect of AIF, F(13,337) « 16.29, £ < .001. However, neither 
the effect of level of training nor the AIF by level of training 
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Table 9» T-4 Trainer: The Number of lEUs (Transition 
and Operational) Indicating the Operational 
Status of Each AIF 



' No such Never ~~ ' 

Feature capability operated Unreliable Reliable 





T 


g 


T 


0 


T 


0 


T 


g 


IT 


19 


7 


0 


0 


0 


0 


1 


1 


R 


1 


0 


0 


0 


0 


0 


19 


a 


TSF 


0 


0 


0 


0 


0 


0 


20 


8 


PF 


18 


7 


0 


0 


0 


0 


2 


1 


RB 


6 


3 


6 


1 


3 


0 


5 


4 


D 


16 


5 


1 


0 


0 


0 


3 


3 


RP 


16 


7 


2 


0 


0 


0 


2 


1 


AMI 


8 


4 


0 


1 


1 


0 


11 


3 


HC 


20 


8 


0 


0 


0 


0 


0 


0 


PTC 


0 


0 


0 


0 


0 


0 


20 


8 


MTC 


0 


0 


0 


0 


0 


0 


20 


8 


PRM 


4 


3 


0 


0 


0 


0 


16 


5 


PAM 


4 


2 


0 


0 


0 


0 


16 


6 


EWS 


20 


8 


0 


0 


0 


0 


0 


0 
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Table 10, T-4 Transition Training: Mean Ratings (and Standard 
Deviations) of the Frequency of Use, Ease of Use, 
Adequacy of Training Received, Training Value, and 
Potential Training Value 





rKtlJUot 




i KttU\ i ; 


1 KttU \C ) 


1 VALUt 


DT\/n I 1 fC 


IT 












4.0 
(1.7) 


R 


5.4 
(1.0) 


5.7 
(0.9) 


6.8 
(0.5) 


6.4 
(0.8) 


6.3 
(1.2) 


6.0 
(1.1) 


TSF 


6.0 
(0.9) 


6.2 
(0.5) 


6.8 
(0.5) 


6.4 
(0.8) 


6.6 
(0.8) 


6.6 
(0.8) 


PF 












4.0 
(2.0) 


RB 


1.2 
(0.4) 


4.4 
(1.7) 


3.9 
(2.5) 


5.0 
(2.2) 


2.1 
(1.2) 


3.0 
(1.1) 


0 












4.6 
(1.7) 


RP 












4.6 
(2.0) 


AMI 


4.5 
(1.0) 


5.6 
(0.5) 


5.5 
(1.7) 


5.6 
(1.4) 


5.5 
(1.5) 


4.8 
(1.8) 


HC 












3.2 
(1.9) 


PTC 


5.9 
(1.4) 


6.0 
(1.2) 


6.6 
(0.8) 


6.0 
(1.1) 


6.4 
(0.7) 


6.4 
(0.7) 


MTC 


6.0 
(1.1) 


4.7 
(1.1) 


6.4 

(0.7) 


6.0 
(1.3) 


6.7 
(0.6) 


6.6 
(0.6) 


PRM 


6.6 
(0.9) 


4.9 
(1.4) 


6.1 
(0.8) 


5.5 
(1.5) 


6.0 
(1.5) 


6.0 
(1.4) 


PAM 


6.8 

(0,8) 


5.2 
(0.8) 


5.8 
(1.2) 


5.7 
(1.0) 


6.0 
(1.3) 


5.6 
(2.2) 


EWS 












4.2 
(1.9) 




5.4 
(1.9) 


5.4 
(1.2) 


6.2 
(1.3) 


5.9 
(1.3) 


5.8 
(1.7) 


5.0 
(1.9) 



o 

ERIC 
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Table 11 . T-4 Operational Training: Mean Ratinas (and Standard 
Deviations) of the Frequency of Use, Ease of Use, 
Adequacy of Training Received, Training Value, 
and Potential Training Value 



heature 


FREQU5E 




TRE(!D (1) 


TRECCi (2) 


TVAlue 


PTVALU 


IT 












3.8 
(1.6) 


R 


5.5 
(1.2) 


5.9 
(0.4) 


6.5 

(0.8) 


6.5 

(0.8) 


6.6 

(0.5) 


6.5 
(0.5) 


TSF 


6.1 
(1.1) 


6.4 

(0.5) 


6.9 
(0.4) 


6.9 
(0.4) 


6.7 
(0.5) 


6.4 
(0.9) 


PF 












4.1 
(2.5) 


RB 


2.5 
(1.0) 


4.8 

(1.3) 


6.0 
(1.4) 


7.0 
(0.0) 


6.0 
(0.8) 


4.9 
(1.4) 


D 












5.9 
(1.2) 


RP 












6.1 
(1.0) 


AMI 


4.3 
(3.1) 


4.7 
(1.2) 


7.0 
(0.0) 


6.5 
(0.7) 


6.5 
(0.6) 


5.5 
(1.7) 


HC 












4.1 

(2.2) 


PTC 


6.2 
(1.2) 


5.9 
(2.1) 


6.6 
(0.5) 


*6.4 

(1.1) 


6.8 
(0.5) 


6.8 
(0.5) 


MTC 


6.1 
(0.8) 


6.0 
(0.5) 


6.4 

(0.5) 


G.5 
(0.5) 


6.6 

(0.5) 


6.9 
(0.4) 


PRM 


6.6 
(0.9) 


5.2 
(1.1) 


6.0 
(1.0) 


5.6 
(1.3) 


6.3 

(1.2) 


6.0 
(2.1) 


PAM 


6.7 
(0.8) 


- 6.0 
(1.3) 


6.2 
(0.8) 


6.0 
(0.6) 


6.5 

(0.8) 


5.5 
(2.3) 


EWS 












4.4 

(2.1) 




5.5 
(1.8) 


5.8 
(1.2) 


6.5 

(0.7) 


6.4 

(0.8) 


6.6 

(0.7) 


5.5 

(1.8) 



interaction was significant, F(l,26) » 3.13, £ > .05 and F(13,337) = 1.15, 
p > .CS, respectively. Potential usefulness was fairly high overall, 
progranwed and manual threat control received the highest ratings, whereas 
instructor tutorial, partial freeze, recorded briefing, hard copy, and 
electronic warfare performance scoring received only moderate ratings 
(Tukey HSO qj « 1.52). 

Interrelations among utilization and utility ratings . Table 12 shows 
the intercorrelations among the ratings of each feature on each of the five 
questions. Most of the coefficients were positive and significant. The 
matrix suggests, for example, that the more useful a feature was, the more 
frequently it was used, the easier it was to use, the more adequate was the 
training in its use, and the greater was its potential training value. It 
should be noted, however, that not all variables were significantly 
correlated. For example, although ease of use and adequacy of training 
received were positively correlated, neither variable was related to 
frequency of use. The matrix reflects considerable variability in the 
level of predictability among the variables. The coefficients of 
determination ranged from .00 (FREQUSE/EASEUSE) to .53 (TVALUE/PTVALUE) . 



Table 12. T-4 Trainer: Matrix of Intercorrelations Among 

Frequency of Use, Ease of Use, Adequacy of Training 
Received, Training Value, and Potential Training Value 



Feature FREQUSE EASEUSE TRECD (1) TRECD (2) TVALUE Pi VALUE 



FREQUSE 


1.00 










EASEUSE 


.06 


1.00 








TRECD(l) 


.13 


.32** 


1.00 






TRECD (2) 


-.05 


.36** 


.46** 


1.00 




TVALUE 


.46** 


.17* 


.40** 


.29** 


1.00 


PT VALUE 


.50** 


.16* 


.21** 


.23** 


.73** 



* £ < .05. 

** £ < .01. 



Table 13 summarized the results of a multiple linear regression 
analysis in which the frequency of AIF use was predicted from a linear 
coirtiination of ease of use, adequacy of training received, and training 
value. The table indicates that, together, the predictor variables 
accounted for 28X of the variability in the frequency-of-use ratings. 
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Table 13. T-4 Trainer: Multiple Linear Regression of 
Frequency of Use on Ease of Use, Adequacy of 
Training Received, and Training Value 



DEPENDENT VARIABLE: Frequency of AIF Use 

MULTIPLE R: .53 STD. ERROR OF EST.: 1.28 

MULTIPLE R-SQUARE: .28 

ANALYSIS OF VARIANCE: 





Sum of Squares 


df 


Mean Sguare F-Ratio 




REGRESSION 


117.9240 


3 


39.3080 23.954 


.0000 


RESIDUAL 


297.0165 


181 


1.6410 




Predictor 
variable 


Coefficient 


Standard 
error 


Standard 
regression 
coefficient t 


£ 


EASEUSE 


.0400 


.0934 


.0294 .428 


.6690 


TRECD 


-.2972 


.1225 


-.1757 -2.426 


.0163 


TVALUE 


.7720 


.0917 


.5679 8.419 


.0000 


(CONSTANT) 


2.5259 


.7709 







Training value was clearly the niost important predictor. However, adequacy 
of training also contributed significantly to the equation (even though it 
was negatively related to the dependent variable). Ease of use did not 
contribute significantly. 

Comments . The T-4 ATD was one of the most highly regarded devices 
surveye?^ In fact, many lEWs preferred it to the more sophisticated B-52 
Weapon Syster Trainer (B-52 WST; see below). One frequent criticism was 
that modifications in T-4 hardware and software have not kept pace with 
those of the aircraft. 

B'Od Weapon System Trainer ■ Defensive Stations 

Training niission. The B-52 Weapon System Trainer (WST) is one of the 
most wpnisticated AlDs in tne Air Force inventory (Stein, 1984). Tt can 
provide training for the entire six-man B-52 crew via three separate 
instructor consoles: the flight instructors console (pilot, co-pilot), the 
navigation Instructors console (navigator, radar navigator), and the 
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defensive avionics systems instructors console (electronic warfare officer, 
gunner) . 

The WST can operate in either independent or integrated mode, depending on 
whether one or all of the crew stations are active at a given time. The 
integrated capability mokes the WST particularly useful for training crew 
coordination* The typical independent WST training session (defensive) is 
similar to that of the 7-4 except that the WST has more sophisticated threat 
library, weapons, real-time randomness, maneuvering, monitoring and mission 
generation capabilities* The integrated session, which includes briefing, 
mission, and debriefing, may last up to five hours. The independent mission 
nonnally lasts up to two hours. 

The emergency war order (EWO) mission provides a context for both 
transition and operational (continuation) training. The integrated EWO 
mission includes equipment checks and malfunction evaluations, takeoff, air 
refueling, high-level flight, descent into enemy territory, defense of 
aircraft in various encounters (airborne interceptors, ships, land-based 
threats), monitoring of EW equipment, ECM, malfunction analyses, climb and 
withdrawal, and landing. During independent missions, the instructor's ^se 
multiple malfunctions, freezes, and resets in order to work on specific 
problem areas (e.g., malfunction analyses, signal recognition, jamming). 

Training of Sis . On the average, formal classroom instruction 
accounted for only 8% of the initial training for the RTU Sis (SD = 11%) 
and 25X for the initial training of the CTU Sis (SD = 29%). Only one WST 
SI reported having received any refresher training. 

AIFs. The operational status of each AIF on the B-52 WST is 
summarTIed in Table 14. There appear to be operational problems associated 
with all available features except auto malfunction insertion. The least 
reliable AIFs (i.e., those AIFs that were called "unreliable" by at least 
0.30 of the Sis) were record/playback (0.73), programmed threat control 
(0.33), procedures monitoring (0.53), and electronic warfare performance 
scoring (0.53). 

Utilization and utility relations . The B-52 WST RTU and CTU Sis' 
ratings are summarized in Tables 15 and 16, respectively. Means and 
standard deviations are listed for available features under Questions 1 to 
4 and for all features under Question 5 (potential usefulness). The 
multivariate analysis of variance revealed a significant overall effect of 
AIF, F(45,432.53) - 4.85, £ < .001. Univariate Fs (df » 9,100) were 
signiTicant for frequency of use (£ < .001), easi of use (£ < .001), and 
training value (p < .001), but not for adequacy of training received (£ < 
.05). Although the multivariate effect of level of training was not 
significant, F(5,3) « 1.05, p < .05, there was a weak but sianificant AIF 
by level of training interaction, F(45,432.53). 1.47, (£ < .()5). 



Frequency of AIF use was lower overall than was the case for the T-4. 
The average feature was used approximately once each mission. Threat 
control (programmed and manual) and parameters monitoring were used mst 
often (at least two to four times a mission), whereas record/playback 
(called "unreliable** by 73% of the Sis), hard copy (called "unreliable" by 
20% of the Sis), and partial freeze were used significantly less often 
(Tukey HSD ^| « 2.48). Ease of use was fairly high overall. Programmed 
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Table 14. B-52 WST-Defens1ve Station: The Number of 
Sis (Transition and Operational) Indicating 
the Operational Status of Each AIF 



No such Never 
F eature capability operated Unreliable Reliable 

I — T 1 — 5 1 — T~ 

IT 76 00 10 10 

R 0 0 1 0 2 1 6 5 

TSF 00 00 00 96 

PF 00 00 1086 

RB 86 10 0000 

D 7 111 110 3 

RP 2 0 1 0 6 5 0 1 

AMI 00 00 00 96 

HC 1000 2165 

PTC 00 00 41 55 

MTC 00 00 31 65 

PRM 20 00 62 14 

PAM 10 00 22 64 

EWS 00 20 44 32 
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Table 15, B-52 WST Transition Training: Mean Ratings (and 
Standard Deviations) of the Frequency of Use 
Ease 6f Use, Adequacy of Training' Received, 
Training Value, and Potential Training Value 



Feature 


FREQUSE 


EASEUSE 


TRECD(l) 


TRECD{2) 


TVALUE 


PTVALUE 


IT 












4.3 














(1.7) 


R 


4.0 


5.1 


4.4 


4.6 


4.9 


6.1 




(l.S) 


(1.1) 


(1.9) 


(1.7) 


(1.8) 


(0.6) 


TSF 


5.1 


6.1 


6.6 


6.1 


6.6 


6.7 




(0.6) 


(0.6) 


(0.5) 


(1.3) 


(0.7) 


(0.5) 


PF 


3.2 


5.9 


5.5 


5.1 


6.0 


6.4 




(1.3) 


(0.8) 


(1.5) 


(1.5) 


(1.0) 


(0.5) 


RB 












3.8 














(1.6) 


D 












5.2 














(1.3) 


RP 




2.8 


3.4 


3.6 


3.2 


5.8 




(O.A) 


(1.1) 


(1.8) 


(2.1) 


(2.1) 


(0.7) 


AMI 


5.1 


5.7 


4.5 


5.6 


5.9 


6.2 




(1.1) 


(1.3) 


(2.4) 


(1.9) 


(0.9) 


(0.7) 


HC 


2.2 


4.4 


5.0 


4.6 


3.1 


5.2 




(0.7) 


(1.8) 


(2.1) 


(1.9) 


(1.5) 


(1.6) 


PTC 


5.1 


6.1 


5.8 


5.1 


5.8 


6.8 




(2.1) 


(1.3) 


(1.7) 


(2.0) 


(1.4) 


(0.4) 


MTC 


5.8 


4.1 


5.2 


4.9 


6.6 


7.0 




11. o) 


10.9) 


U.O) 




lu./J 




PRM 


5.0 


5.2 


5.1 


4.6 


3.0 


4.7 




lo. 1 J 


M f:\ 
U .oj 


to 1 \ 




1 1 . / i 


10 0) 


PAM 


6.5 


4.4 


5.7 


5.6 


4.5 


5.2 




(0.9) 


(1.7) 


(1.0) 


(1.4) 


(1.5) 


(2.2) 


EUS 


2.9 


5.1 


5.2 


4.8 


3.8 


5.8 




(1.9) 


(1.6) 


(2.0) 


(1.6) 


(1.8) 


(1.3) 




4.3 


&.0 


5.Z 


5.0 


5.0 


5.7 




(2.1) 


(1.5) 


(1.9) 


(1.7) 


(1.8) 


(1.5) 
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Table 16 , B-52 WST Operational Training: Mean Ratings (and 
Standard Deviations) of the Frequency of Use, 
Ease of Use, Adequacy of Training Received, and 
Potential Training Value 



heature FREQUSE EASEUSE TftECO(l) TftgCD(2) TTCuE — PTvAluE 



IT 












4.8 
(1.9) 


R 


(1.2) 


5.3 
(1.4) 


6.2 
(0.8) 


5.7 
(2.0) 


5.7 
(1.8) 


6.2 
(1.6) 


TSF 


4.7 
(1.4) 


6.7 

(0.5) 


6.7 
(0.5) 


6.2 
(1.2) 


6.0 
(2.0) 


6.7 
(0.5) 


PF 


3.2 
11.5 ) 


6.5 
(0.5) 


5.8 
(1.2) 


E^.5 
(1.4) 


4.5 

(2.2) 


5.5 
(2.1) 


RB 












4.2 
(1.8) 


D 












4.5 
(1.9) 


RP 


1.0 
(0.0) 


3.2 
(1.5) 


3.2 
(1.8) 


3.0 
(2.5) 


1.7 
(1.2) 


4.7 
(1.8) 


AMI 


4.0 

(2.7) 


6.5 
(0.8) 


6.5 
(0.8) 


5.5 
(2.1) 


5.5 
(2.0) 


6.8 
(0.4) 


HC 


2.0 
(1.5) 


4.5 

(0.8) 


5.5 
(0.8) 


5.0 
(1.7) 


2.5 
(1.5) 


4.3 
(2.3) 


PTC 


6.7 
(0.8) 


6.7 
(0.5) 


6.2 
(1.2) 


5.3 
(2.0) 


6.2 
(0.8) 


6.7 
(0.8) 


MTC 


5.3 
(0.8) 


4.3 
(1.4) 


6.2 
(0.8) 


6.3 
(1.2) 


6.5 
(0.8) 


6.7 
(0.5) 


PRM 


4.7 
(2.9) 


3.2 
(1.8) 


5.0 
(1.8) 


4.3 
(2.2) 


4.3 

(2.4) 


6.2 
(1.6) 


PAM 


5.8 
(2.0) 


4.3 
(1.4) 


5.7 
(1.0) 


5.2 
(1.2) 


6.2 
(1.3) 


6.3 
(1.6) 


EWS 


4.2 
(2.6) 


3.8 
(1.9) 


5.3 
(2.3) 


5.2 
(1.9) 


3.0 
(2.3) 


5.3 
(2.3) 



4.2 5.0 5.5 5.1 4.5 5.6 

(1.7) (1.8) (1.6) (2.0) (2.3) (1.8) 
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threat control, auto malfunction Insertion, and freeze were rated "very 
easy," wiiereas record /playback was rated "fairly difficult" (Tukey HSD > 

1.72). The training received by the Sis was apparently adequate despite 
the lack of both formal classroom Instruction and refresher training. (See 
previous section.) Threat control (progranmed and manual), auto 
malfunction Insertion, and total system freeze were judged to have the most 
training value, whereas record/playback and hard copy were rated as only 
"fairly useful" (Tukey HSO qj . 2.00). 

The separate analysis of the potential training value ratings revealed 
a significant effect of AIF, F(13»169) • 6.72, p < .001. However, neither 
the effect of level of training nor the AIF by Tevel of training 
Interaction was significant, F < 1.00 and F(13cl69) » 1.25, £ < .05, 
respectively. Potential usefulness was faTrly high overall. In fact. It 
was somewhat higher than the usefulness of the AIFs as they are currently 
Implemented (Question 4). Extremely high ratings were assigned to threat 
control (programmed and manual), auto malfunction Insertion, and total 
systt freeze. Even the lowest rated features (e.g.. Instructor tutorial, 
recorjed briefing) were Judged to be at least moderately useful. (Tukey 
HSD • 1.86). 

Interrelations among utilization and utility ratings . Table 17 shows 
the Intercorreiations among the ratings of each feature on each of the five 
quest lens. Most of the coefficients were positive and significant; the 
only exceptions were the correlations of potential training value with ease 
of use and adequacy of training received. The matrix reflects a low, 
although variable, level of predictability among the variables. The 
coefficients of determination ranged from .00 (PTVALUE/TRCCD(2)) to .23 
(TRECD(1)/TRECD(2)). 

Table 17. B-52 WST: Matrix of Intercorreiations Among 
Frequency of Use, Ease of Use, Adequacy of 
Training Received, Training Value, and 
Potential Training Value 



Feature 


FREQUSE 


EASEUSE 


fA^eb(l) 


TRECD(2) TVALUE PTVALUE 


FREQUSE 


1.00 








EASEUSE 


.22* 


1.00 






TRECD(l) 


.30* 


.50* 


1.00 




TRECD(2) 


.30* 


.45* 


.53* 


1.00 


TVALUE 


.49* 


.41* 


.32* 


.52* 1.00 


PTVALUE 


.35* 


.16 


.13 


.06 .45* 1.00 



* £ < .01. 
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Table 18 sumrwirizes the results of a multiple linear regression 
analysis in v/hich the frequency of AIF use was predicted from a linear 
combination of ease of use, adequacy of training receivevi, and training 
value. The table indicates that, together, the predictor variables 
accounted for approximately 29% of the variability in the frequency-of-use 
ratings. Training value was clearly the most important predictor. 
However, adequacy of training received also contributed significantly to 
the equation. Ease of use did not. 



Tabl e 18, B-52 WST: Multiple Linear Regression of 

Frequency of Use on Ease of Use, Adequacy of 
Training Received, Training Value, and 
Potential Training Value 



JCE^ frequency^of^ATFlIse 
MULTIPLE R: .53 STO. ERROR OF EST.: 1.82 

MULTIPLE R-ngUARE: .29 



ANALYSIS OF VARIANCE: 





Sum of Squares 


df 


Mean Square 


F-Ratio 




REGRESSION 


189.0932 


3 


63.0311 


18.932 


.0000 


RESIDUAL 


472.7766 


142 


3.3294 






Predictor 
variable 


Coefficient 


Standard 
error 


Standard 

regression 

coefficient 


t 




EASEUSE 


-.0724 


.1112 


-.0561 


-.651 


.5159 


TRECD 


.2823 


.1222 


.2012 


2.311 


.0223 


TVALUt 


.6723 


.1175 


.4496 


5.722 


.0000 


(CONSTANT) 


-.4946 


.6967 









Conments. It was noted above that many WST AIFs were described as "un- 
reliable." There were sevsi^al other criticisms of the device: certain threats 
were said to be "out of date," thereby limiting effective ECM training. 
Moreover, the WST generates threats probabilistically. Some repeatability will 
be included with the threat update effort now in progress. Thus, it is not 
possible to replicate a particular threat scenario exactly. Another apparent 
problem is that feedback data, which are presented to the Sis on 
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multiple CRT "pages," are not arranged for rapid access during dynamic 
situations. 

FB-lllA Operational Flight Trainer 

The FB-lllA radar navigator Instructors' data were collected during 
Phase II. Those results are summarized In Appendix B (Tables B 1 through 
B-5). The reader Is cautioned that, due to differences between the Phase 
II and Phase III questionnaires, the F6-111A results and those of Phase III 
are not directly comparable. See Polzella (1985) for additional 
Information regarding the utility and utilization of the FB-lllA 
Operational Flight Trainer AIFs. 

Tactical Air Command 

F-46 Simulator 

Training mission . The F-46 Advanced Wild Weasel is the ultimate USAF 
version or tne F-4 pnantom II fighter aircraft. In addition to its 
offensive capabilities, the F-4e carries ECM sensors, jamming pods, chaff 
dispensers, anti-radiation missiles, and advanced avionics. The typical F- 
46 simulator mission required students to demonstrate the various system 
capabilities within a highly dynamic scenario. The RTU and CTU missions 
are similar; however, CTU missions are characterized by more elaborate 
threat scenarios, whereas RTU missions tend to stress basic tactical 
skills. 

A typical training session consists of a 15-m1nute briefing of the 
planned mission elements, a 1 1/2-hour mission, and 15 to 30 minutes for 
debriefing and critique. The major mission segments are preflight, 
takeoff. Ingress into EH/target area, egress, and return to base. Mission 
elements normally Include new APR-38 functions, surface-to-air missile 
(SAM) encounters, air-to-air Intercepts, ordnance delivery, and a full 
spectrum of EU activity. In addition, there are frsquent malfunctions, 
emergencies, and threats. F-46 Sis prefer to Insert these manually and use 
freeze and reset as needed. 

Training of Sis . On the average, formal classroom instruction 
accounted for 3g» of the initial training for the RTU SIS (SO « 34X) and 
15X of the initial training for the CTU Sis (SO - 22X). Only two F-4G Sis 
(1 RTU, 1 CTU) reported having received any refresher training. 

AIFs . The operational status of each AIF on the F-46 simulator is 
summarized in Table 19. Except for hard copy, there appear to be few 
operational problems associated with the F-46 features. However, 
substantial proportions of Sis apparently have never operated partial 
freeze (0.47), auto malfunction insertion (0.84), and hard copy (0.53). 
(Note: The proportions include those Sis who indicated "no such 
capability" for these features.) 

Ut11l7<t1on and utility ratings . The ratings for the F-4G RTU and CTU 
Sis are summarized in raoies zu ana 21, respectively. Means and standard 
deviations are listed for available features under Questions 1 to 4 and for 
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Table 19. F-46 Simulator: The Number of lEWs (Replacement 
and Operational) Indicating the Operational Status 
of Each AIF 



No such Never 
Feature capability operated Unreliable Reliable 





R 


0 


R 


0 


R 


0 


R 


0 


IT 


12 


17 


0 


1 


1 


0 


0 


1 


R 


3 


0 


0 


0 


0 


1 


10 


18 


TSF 


0 


0 


0 


0 


0 


0 


13 


19 


PF 


8 


7 


0 


0 


0 


2 


5 


10 


RB 


13 


19 


0 


0 


0 


0 


0 


0 


0 


13 


18 


0 


1 


0 


0 


0 


0 


RP 


13 


15 


0 


4 


0 


0 


0 


0 


AMI 


12 


12 


0 


3 


0 


0 


1 


4 


HC 


5 


2 


4 


6 


4 


5 


0 


6 


PTC 


1 


1 


4 


1 


0 


3 


8 


14 


HTC 


0 


0 


0 


0 


0 


0 


13 


19 


PRM 


13 


16 


0 


0 


0 


0 


0 


3 


PAM 


12 


16 


0 


0 


1 


1 


0 


2 


EUS 


12 


14 


1 


2 


0 


1 


0 


2 



30 



107 



ERIC 



Table 20, F-4G Simulator Replacement Training: Mean 
Ratings (and Standard Deviations) of the 
Frequency of Use, Ease of Use, Adequacy of 
Training Received, Training Value, and 
Potential Training Value 



Feature 


FREQUSE 


EASEUSE 


TRECD(l) 


TRECd(2) 


TVALUE 


PTVALUE 


IT 












4.4 

(1.8) 


R 


4.0 
(1.4) 


5.5 
(0.9) 


5.2 
(1.9) 


5.2 

(1.0) 


5.5 
(0.9) 


6.0 
(0.8) 


TSF 


4.2 
(1.5) 


6.0 
(0.4) 


5.8 
(1.1) 


5.3 

(1.0) 


5.4 
(1.0) 


5.8 
(1.1) 


PF 


4.4 

(0.9) 


5.0 
(1.7) 


6.0 
(0.6) 


5.0 
(1.7) 


5.6 
(1.1) 


4.6 

(2.1) 


RB 












3.5 
(2.0) 


D 












4.4 

(1.8) 


RP 












5.5 
(1.5) 


AMI 












4.3 
(1.7) 


HC 


2.3 
(1.8) 


4.2 
(1.6) 


4.2 

(2.1) 


4.5 
(1.3) 


3.6 
(1.5) 


4.6 
(1.1) 


PTC 


3.0 
(1.8) 


5.7 
(1.2) 


5.3 
(1.3) 


4.7 

(1.2) 


4.5 

(2.0) 


5.4 
(1.2) 




0 .D 

(0.7) 


A 0 

(1.0) 


A Q 

(1.6) 


L A 
H .o 

(1.4) 


o • 0 

(1.1) 


(0.7) 


PRM 












<v 8 
(1.4) 


PAM 












5.2 
(1.2) 


EWS 












5.0 
(2.0) 




4.0 
(2.0) 


5.2 
(1.0) 


5.3 
(1.5) 


5.0 
(1.2) 


5.2 
(1.6) 


5.0 

(1.6) 
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Table 21. F-4G Simulator Operational Training: Mean 
Ratings (and Standard Deviations) of the 
Frequency of Use, Case o'f Use, Adequacy of 
Training Received, Training Value, and Potential 
Training Value 



Feature 


FREOUSE 


EASEUSE 


TRECD(l) 


TRECD(2) 


tVALUt " 




IT 












4.4 

(1.4) 


R 


4.9 

(0.9) 


6.0 

(0.9) 


6.5 

(0.6) 


6.2 
(0.9) 


6.1 

(1.0) 


6.4 
(0.6) 


TSF 


4.2 
(1.1) 


6.3 
(0.5) 


6.7 

(0.5) 


6.4 
(0.8) 


6.3 
(0.7) 


6.5 
(0.7) 


PF 


3.9 
(1.4) 


4.9 

(1.2) 


5.5 
(1.8) 


5„2 
(2.1) 


5.1 
(1.6) 


5.4 
(1.5) 


RB 












2.9 
(1.7) 


D 












4.6 
(1.7) 


RP 












5.3 
(1.6) 


AMI 


2.0 
(1.7) 


4.8 
(2.0) 


4.0 

(2.3) 


4.0 
(2.9) 


2.8 
(2.1) 


4.4 

(1.8) 


HC 


2.0 
(1.5) 


4.1 

(M) 


5.3 
(1.9) 


3.9 
(2.2) 


3.7 
(1.7) 


5.2 
(1.3) 


PTC 


4.1 

(1.7) 


5.9 

(1.1) 


5.5 
(1.5) 


5.6 
(1.7) 


5.1 
(1.4) 


5.4 
(1.3) 


MTC 


5.6 

(1.2) 


4.3 

(1.1) 


5.5 
(1.6) 


5.4 
(1.8) 


6.4 

(0.8) 


6.5 
(0.5) 


PRM 












4.7 
(1.9) 


PAH 












5.0 
(1.7) 


EWS 












5.2 
(1.8) 




4.0 

(1.8) 


5.1 
(1.6) 


5.7 

(1.7) 


5.4 
(1.8) 


5.2 
(1.8) 


5.1 
(1.7) 
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an features under Question 5 (potential usefulness). The multivariate 
analysi* of variance revealed a significant overall effect of AIF, F(25,350.7) 
8.32, £ < .001. Each univariate £ (df = 5,98) was also significant, £< 
.001. The analysis also revealtv' a significant overall effect of level of 
training, F(5,l) « 1458.03, £ < .u5; however, none of the univariate Fs 
were sianiTicant in this case, £< .05. Finally, there was a significant 
AIF by level of training interaction, £(25,350.7) = 1.97, £ < .01. 
Univariate interactions were significant for frequency of use (£ < .05), 
and training value (£ < .05). 

A moderate level of frequency of AIF use was observed overall. The 
average feature was used only once each mission. Manual threat control was 
used with considerable regularity (five to seven times each mission), 
whereas hard copy (called "unreliable" by 28X of the Sis) and auto 
malfunction insertion were rarely or never used (Tukey HSD qj = 1.30). The 

AIF by level of training interaction apparently reflected the fact that RTU 
Sis used manual threat control more often than they used all other 
features, whereas CTU (i.e., operational) Sis used manual threat control, 
programmed threat control, reset, and freeze (total and partial) at 
statistically equivalent rates (Tukey HSD Qg = 1.93). While there appeared 

to be no particular difficulties in using any of the features, some 
features (e.g., reset, total system freeze) were rated easier to use than 
others (e.g., hard copy, manual threat control), Tukey HSD qj = 1.12. The 

RTU and CTU Sis rated the training they received (both in the operation and 
effective use of the features) as adequate (Tukey HSD qj = 1.19,1.29). It 

is interesting to note that the ratings of the RTU Sis were somewhat lower 
than those of the CTU Sis, despite the fact that the RTU Sis apparently 
received more formal training. (See "Training of Sis.") The training 
value ratings were fairly high overall. The average featu-e was judged 
"very useful." The highest ratings were assigned to manual threat control, 
reset, and total system freeze. Auto malfunction insertion and hard copy 
received significantly lower ratings than did all other features (Tukey HSD 
Qj « 1.07). The significant AIF by level of training interaction was 

apparently due to a greater degree of variability among the CTU Sis' 
ratings than among those of the RTU Sis (Tukey HSD Qg = 1.59). 

The separate analysis of the potential training value ratings revealed 
a significant effect of AIF, F(13,387) = 13.85, £ < .001. However, neither 
the effect of level of training nor the AIF by level of training 
interaction was significant, F < 1.00 and F < 1.00, respectively. 
Potential usefulness was fairly high overall. Every feature was judged to 
be at least fairly useful. Manual threat control, reset, and total system 
freeze received the highest ratings, whereas instructor tutorial, recorded 
briefing demonstration, and auto malfunction insertion received 
significantly lower ratings (Tukey HSD qj = 1.26). 

Interrelations among utilization and utility ratings . Table 22 shows 
the intercorreiatlons among the ratings of each feature on each of the five 
questions. Most of the coefficients were positive and significant. 
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Table 22. F-4G Simulator: Matrix of Intercorrelations 

Among Frequency of Use, Ease of Use, Adequacy of 
Training Received, Training Value, and Potent lal 
Training Value 



Feature 


FRCtUSE 




TRECD(l) 


TRECD{2) 


TVALUE 


PTVALUt 




1 00 












EASEUSE 


.04 


1.00 










TRECD(l) 


.15 


.45** 


1.00 








TKSCD(2) 


.28** 


.39** 


.64** 


1.00 






TVALUE 


.71** 


.19* 


.30** 


.37** 


1.00 




PTVALUE 


.52** 


.08 


.22** 


.30** 


.70** 


1.00 



*£ < 



.05. 

.01. 



Interestingly, however, neither the adequacy of training received (in 
operating the features) nor the ease of using the features was related to 
frequency of use. Overall, *,he matrix reflects a variable level of 
predictability among the variables. The coefficient of determination 
ranged from .00 (FREQUSE/EASEUSE) to .50 (FREQUSE/TVALUE) . 

Table 23 summarizes the results of a multiple linear regression 
analysis In which the frequency of AIF use was predicted from a linear 
combination of ease of use, adequacy of training received, and training 
value. The table Indicates that, together, the predictor variables 
accounted for approximately 47X of the variability In the frequency-of-use 
ratings. However, the only significant predictor was training value. 

Comments . There were numerous criticisms of the F-4G simulator, most 
of which related to fidelity and operational reliability. Apparently, nwst 
instructors are self-taught, and some are not familiar with the full range 
of capabilities. Many Sis characterized the simulator as a "procedural 
trainer" rather than a "full-mission simulator," which It is designed to 
be. Various reconmendatlons were made: record/playback capability, rear 
cockpit visual display, electronic warfare and weapons scoring. Imaging of 
electronic missile launches, and better and more structured Instructor 
training. 

A-10 Simulator 

Training mission .. The A-10 Is a heavily armed (and armored) close air 
support attack aircraft. As presently configured, the A-10 pilot also 
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Table 23. F-4G Simulator: Multiple Linear Regression of 
Frequency of Use on Ease of Use, Adequacy of 
Tiaining Received, and Training Value 

DEPENDENT VARIABLE: Frequency of AlT-Use 

MULTIPLE R: .68 STD. ERROR OF EST. : 1.16 

MULTIPLE R-SQUARE: .47 

ANALYSIS OF VARIANCE: 





Sum of Squares 


df 


Mean Square 


F-Ratio 




REGRESSION 


173.0386 


3 


b7.6795 


43.078 


.0000 


RESIDUAL 


19b. 1654 


148 


1.3390 






Predictor 
variable 


Coefficient 


Standard 
error 


Standard 

regression 

coefficient 


t 


£ 


EASEUSE 


-.0918 


.0788 


-.0788 


-1.165 


.2457 


TRECO 


.0313 


.0824 


.0272 


.379 


.7050 


TVALUE 


.9677 


.0915 


.6815 


10.575 


.0000 


(CONSTANT) 


-.8718 


.5928 









functions as weapon systems officer. In this latter capacity, he is 
responsible for navigation, ECM operations, and target or threat 
acquisition and designation. 

Because of its extensive capabilities, the A-10 simulator has been 
used as both part-task trainer and full-mission simulator. A fully 
integrated mission includes pref light, takeoff, TACAN point-to-point 
through a programmed series of threats, switchology, operation of radar 
warning receiver (RWR), threat recognition, evasion, ECM pod operations, 
chaff /flare usage, weapons delivery, egress, and return-to-base. The major 
difference between the RTU and CTU missions is that CTU Sis devote more 
time to the integrated mission than do RTU Sis, whereas RTU Sis devote 
relatively more time to the training of particular EW techniques. 

Training of Sis . On the average, formal classroom instruction 
accounrea for ZZ» of the A-10 RTU Sis' initial training (SD = 30%) and IIX 
of the CTU Sis' initial training (SD » 16X). Fifty percent of the RTU Sis 
and 35* of the CTU Sis reported having received refresher training (within 



the past 18 weeks and within the past 5 weeks, respectively). More than 
one-third of that training was characterized as formal classroom 
instruction. 

AIFs. The operational status of each AIF on the A-10 simulator is 
summarTIeH in Table 24. Apparently there are operational reliability 
problems associated with all A-10 AIFs except reset and total system 
freeze. In particular, almost one out of four Sis indicated that 
electronic warfare performance scoring was unreliable. The table also 
suggests that a substantial proportion of Sis have never operated 
demonstration (0.97), record /playback (0.39), auto malfunction insertion 
(0.52), manual threat control (0.30), (advanced) procedures monitoring 
(0.48), or (advanced) parameters monitoring (0.52). (Note: The "advanced" 
monitoring capabilities inform the console operator when aircraft 
parameters are "out of bounds." Also note that hard copy is available on 
the A-10 simulator; however, it is not available for copying EW data.) 

Utilization and utility ratings . The ratings for the A-10 RTU and 
CTU Sis are summarized in Tables Z5 and 26, respectively. Means and 
standard deviations are listed for available features under Questions 1 to 
4 and for all features under Question 5 (potential usefulness). The 
multivariate analysis of variance revealed a significant overall effect of 
AIF, F(30),446) = 7.40, p < .001. Each univariate F (df = 6,115) was also 
signiTicant, £ < .001. the multivariate effect of Tevel of training was 
not significant, F(5,l) = 54.69, £ > .05; however, there was a significant 
AIF level of training interaction, £(30,446) = 2.31, £ < .001. 
Significant univariate interactions were found for frequency of use (£ < 
.001), adequacy of training received (£ < .05), and training value 
(£ < .05). 

A relatively low level of frequency of AIF use was observed overall. 
In fact, it was lower for t"ie A-10 ATD than for any other device surveyed. 
The average feature was used only once every two to four missions. 
Demonstration, record/playback, and auto malfunction insertion were rarely 
or never used; however, total system freeze was used with some regularity 
by both RTU and CTU Sis (Tukey HSD qj = 1.52). Moreover, RTU Sis used 

procedures monitoring, parameters monitoring, and electronic warfare 

performance scoring relatively often despite problems in reliability. (See 

previous section.) CTU Sis used these features significantly less often 

(Tukey HSD = 2.10). Ease of use was moderately high for most features. 
.05 

However, significant difficulties in using demonstration, record/playback, 
auto malfunction insertion, and manual threat control were reported by both 
RTU and CTU Sis (Tukey HSD Qg = 1.24). Adequacy of training received (both 

in the operation and effective use of the features) was rated as "slightly 
adequate" overall. Training was most adequate for reset and :otal system 
freeze and least adequate for demonstration, record /pi ay back, auto 
malfunction insertion, and msnual threat control (Tukey HSD qj = 1.31, 

1.35). The adequacy of training received in the operation and use of 
record /pi ay back was significantly lower for RTU Sis than for CIs (Tukey 
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Table 24 . A-10 Simulator: The Number of Sis (Replacement 

and Operational) Indicating the Operational Status 
of Each AIF 



No such Never 
Feature capability operated Unreliable Reliable 





R 


0 


R 


0 


R 


0 


R 


0 


i 1 


14 


16 


1 


1 


1 


0 


0 


0 


K 


1 


0 


2 


0 


0 


0 


13 


17 


lor 


0 


fx 

0 


0 


0 


0 


0 


16 


17 


Pr 


•* 

3 


2 


1 


2 


1 


1 


11 


12 


DD 

Kb 


lb 


1 "7 

17 


1 


0 


0 


0 


0 


0 


n 


7 




9 


11 


0 


0 


0 


1 


RP 


c 


u 


iU 


1 
1 


1 


0 


3 


16 


AMI 


0 


4 


8 


5 


1 


2 


7 


6 


HC 


6 


0 


4 


0 


3 


1 


3 


16 


PTC 


1 


2 


1 


1 


3 


1 


11 


13 


MTC 


5 


0 


2 


3 


1 


2 


8 


12 


PRM 


2 


9 


1 


4 


3 


1 


10 


3 


RAM 


1 


13 


0 


3 


1 


0 


14 


1 


EWS 


0 


1 


0 


1 


6 


2 


10 


13 
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Table 25 , A-10 Simulator Replacement Training: Mean 
Ratings (and Standard Deviations) of the 
Frequency of Use, Ease of Use, Adequacy of 
Training Received, Training Value, and 
Potential Training Value 



Feature 


FREQUSE 


EASEUSE 


TRECDd ) 


TRECD(2) 


TVALUE 


Pi VALUE 


IT 












4.8 














(1 .6 ) 


R 


3.3 


4.7 


b.l 


5.2 


4.1 


5.1 




(1.9) 


(1.3) 


/I A \ 

(1.9 ) 


/I A \ 

(1.9) 


(1 . b) 


(l.o J 


TSF 


4.9 


6.0 


6.4 


6.2 


6.1 


6.2 




/A n \ 

(0.9) 


(0.7) 


/A A \ 

(0.9 ) 


(0.9 ) 


(0.9 J 




PF 


2.1 


4.3 


4.8 


4.7 


4.1 


4.8 




(1.8) 


(1.6) 


(1.8) 


/I A \ 

(1.9) 


(1 .0 ) 


(l.t)) 


RB 












4.1 














( l.o) 


D 


1.0 


2.2 


2.3 


2.0 


1.7 


3.7 




(0.0) 


(1.0) 


(1.5) 


/A A \ 

(0.0) 


/A C\ 

(0.6) 


(^.1 ) 


RP 


1.1 


2.3 


3.1 


3.3 


1.5 


3.7 




(0.3) 


(1.0) 


(1.8) 


(2.1 ) 


/A c ^ 
(O.b) 


( 1 . / j 


AMI 


1.4 


3.2 


4.1 


4.4 


2.8 


4.2 




(1.0) 


(1.6) 


(2.2) 


f 1 A \ 

(1.9) 


(1.7) 


/I Q \ 

(l.o j 


HC 












4.1 
















PTC 


4.3 


4.9 


5.2 


4.8 


3.6 


5.5 




(2.2) 


(2.0) 


(1.7) 


(1.7) 


(2.1) 


(1.5) 


MTC 


3.5 


2.9 


3.7 


4.0 


4.0 


6.2 




(2.4) 


(1.7) 


(1.6) 


(1.8) 


(2.1) 


(0.8) 


PRM 


4.2 


4.8 


5.9 


5.9 


4.3 


4.9 




(2.7) 


(1.8) 


(1.2) 


(1.2) 


(2.2) 


(1.9) 


PAM 


5.9 


4.8 


5.6 


5.3 


4.7 


5.4 




(1.7) 


(1.8) 


(1.3) 


(1.4) 


(1.9) 


(1.8) 


EWS 


6.1 


4.6 


5.7 


5.4 


3.7 


5.9 




(1.7) 


(1.7) 


(1.1) 


(1.4) 


(2.0) 


(0.9) 




3.3 
(2.4) 


4.2 

(1.9) 


5.1 

(1.8) 


5.0 
(1.8) 


3.8 
(2.1) 


4.9 
(1.8) 
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Table 26. A-10 Simulator Operational Training: Mean Ratings 
(and Standard Deviations) of the Frequency of Use, 
Ease of Use, Adequacy of Training Received, Training 
Value, and Potential Training Value 



Feature FREQUSE EASEUSE TRECD(l) TRECD(2) fVALU£ PTVALUE 



IT 












3.8 
(1.7) 


R 


5.1 

(0.6) 


5.9 

(0.7) 


6.2 
(1.4) 


6.3 
(0.8) 


5.8 
(1.3) 


6.2 
(0.9) 


TSF 


5.3 
(0.8) 


6.1 
(0.3) 


6.8 
(0.4) 


6.5 
(0.8) 


6.6 
(0.6) 


6.5 
(0.9) 


PF 


2.7 
(1.6) 


5.1 
(1.4) 


5.7 

11. 1) 


4.6 
(1.6) 


4.0 
(1.5) 


4.6 
(1.4) 


RB 












2.9 

(1.7) 


D 


1.1 
(0.3) 


3.0 
(1.6) 


3.3 
(1.7) 


3.7 
(1.9) 


2.2 
(1.3) 


3.6 
(1.6) 


RP 


2.2 

(KO) 


4.6 
(1.1) 


5.2 
(1.5) 


4.9 

(1.6) 


3.7 
(1.4) 


4.4 

(1.5) 


AMI 


1.5 

(1.2) 


4.1 

(1.5) 


4.1 
(1.8) 


3.8 
(2.3) 


3.2 
(0.9) 


4.0 
(1.2) 


HC 












4.3 
(1.6) 


PTC 


3.0 

(1.7) 


4.5 

(1.6) 


4.3 
(1.6) 


3.6 
(2.0) 


3.2 
(1.4) 


4.8 
(1.4) 


MTC 


3.1 
(2.3) 


2.6 
(1.0) 


2.9 
(1.5) 


5.0 
(1.4) 


3.7 
(1.8) 


5.8 

(1.3) 


PRM 


1.3 

(0.5) 


4.0 
(1.6) 


5.3 
(1.2) 


5.0 

(0.7) 


2.7 
(2.0) 


4.1 

(1.7) 


PAM 


2.2 
(2.7) 


6.0 
(0.0) 


4.5 

(3.5) 


5.7 
(1.2) 


2.5 
(2.4) 


4.1 

(2.0) 


EWS 


3.3 
(2.2) 


4.3 
(1.6) 


4.7 
(1.6) 


4.4 

(1.8) 


3.8 

(1.8) 


4.6 

(2.0) 




3.0 
(1.9) 


4.7 
(1.6) 


EJ.l 
(1.8) 


5.0 

(1.8) 


4.1 

(1.9) 


4.5 
(1.8) 



ERIC 
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HSD ftc« 1.81,1.87). The AIFs, as presently Implemented, were rated 

"nwderately useful" overall. In general, the training value ratings were 
lower for the A-10 ATO than for any other device surveyed. The most useful 
features were reset and total system freeze, whereas demonstration, 
record/playback, and auto malfunction Insertion were judged to be 
significantly less useful (Tukey HSD » 1.41). RTU Sis rated parameters 

monitoring significantly higher In usefulness than did CTU Sis, whereas CTU 
Sis rated record/playback significantly higher In usefulness than did RTU 
Sis. 

The separate analysis of the potential training value ratings revealed 
a significant main effect of AIF, F(13,400) • 13.11, P < .001 and a 
s1gi>1f1cant AIF by level of training Interaction, F(13,400) « 2.42, p < 
.01. The main effect of level of training was nonsignificant, F(l,31) » 
1.5', B > .05. Potential usefulness was moderately high overall. Reset, 
total system freeze, manual threat control, and electronic warfare 
performance scoring received the highest ratings, whereas Instructor 
tutorial, recorded briefing, demonstration, record/playback, auto 
malfunction insertion, and hard copy received significantly lower ratings 
(Tukey HSD « 1.30). The AIF by level of training interaction was 

apparently due to a greater degree of variability among the CTU Sis' mean 
ratings than among those of the RTU Sis (Tukey HSD Qg ■ 1.62). 

Interrelations among utilization and utility ratings . Table 27 shows 
the intercorrelations among the ratings of each feature on each of the five 
questions. Except for the correlation between potential training value and 
ease of use, all of the coefficients were positive and significant. In 
general, the level of predictability was moderate. The coefficient of 
determination ranged from ,02 (EASEUSE/PTVALUE) to .45 (EASEUSE/TRECDd) ). 



Table 27. A-10 Simulator: Matrix of Intercorrelations 
Among Frequency of Use, Ease of Use, Adequacy 
of Training Received, Training Value, and 
Potential Training Value 



o 
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Feature 


FREQUSE 


b^bUSh 


TRECD(l) 


REC0(2) TVALUE PTVALUE 


FREQUSE 


1.00 








EASEUSE 


.25** 


1.00 






TRECO(l) 


.25** 


.67** 


1.00 




TRECD(2) 


.31** 


.47** 


.51** 


1.00 


TVALUE 


.40** 


.55** 


.44** 


.62** 1.00 


PTVALUE 


.48** 


.13 


.18** 


.40** .51** 1.00 




**£ < .01. 
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Table 28 summarizes the results of a multiple linear regression 
analysis In which the frequency of AIF use, adequacy of training received, 
and training value. The table Indicates that, together, the predictor 
variables accounted for only 25X of the variability In the frequency-of-use 
ratings. The only significant predictor was training value. 



Table 28 . A-10 Simulator: Multiple Linear Regression of 
Frequency of Use on Ease of Use, Adequacy of 
Training Received, Training Value, and Potential 
Training Value 



DEPENDENT VARIABLE: Frequency of AIF Use 

MULTIPLE R: .50 STD. ERROR OF EST. : 1.76 

MULTIPLE R-SQUARE: .25 

ANALYSIS OF VARIANCE: 





Sum of Squares 


df 


Mean Square 


F- Ratio 




REGRESSION 


223.3051 


3 


74.4350 


24.067 


.0000 


RESIDUAL 


671.1474 


217 


3,0928 






Predictor 
variable 


Coefficient 


Standard 
error 


Standard 

regression 

coefficient 


t 


£ 


EASEUSE 


.0255 


.1104 


.0181 


.231 


.8173 


TRECD 


.0930 


.1370 


.0585 


.678 


.4983 


TVALUE 


.6892 


.1077 


.4558 


6.399 


.0000 


(CONSTANT) 


-.0152 


.5672 
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Comments . The A-10 simulator was criticized more often than was any 
other ATu surveyed. The most frequent criticisms concerned the device's 
lack of fidelity to the actual aircraft. Several Instructors even believed 
that transfer of training, from simulator to aircraft, was negative. Many 
Sis expressed particular frustration at the limited (RTU) or absent (CTU) 
visual system. Such deficiencies are Ironic when considering that the A-10 
pilot frequently needs visual contact with targets and threats In order to 
fulfill his mission. 

9^ • " 118 
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Several other problems were noted: a lack of instructor confidence in 
the operational reliability of the device, insufficient instructor 
training, and difficulty in manually controlling threats. A few Sis stated 
that the device was useful for procedural training, but many more of the 
comments were critical. 



IV. DISCUSSION 



For purposes of discussion the 14 AIFs surveyed in Ph!:se III can be 
arranged in three categories: 

Briefing AIFs are designed for briefing the student and SI prior to or 
during a training mission. Their purpose is to establish a learning set 
and increase learning readiness. These features include 

1. Instructor tutorial 

2. Recorded briefing 

3. Demonstration. 

Mission Control AIFs include various features designed to control the 
structure and sequencing of tasks within a training mission. These 
features include 

1. Total system freeze 

2. Partial freeze 

3. Reset 

4. Automated malfunction insertion 

5. Programmed threat control 

6. Manual threat control. 

Monitor and Feedback AIFs permit the simulator instructor to monitor 
student performance and provide the student with performance feedback. 
These features Include 

1. Procedures monitoring 

2. Parameters monitoring 

3. Electronic warfare performance scoring 

4. Hard copy 

5. Record/playback. 

Briefing AIFs 

Operational status . Most of the devices surveyed had no briefing 
capability. The only exceptions were the T-5 (recorded briefing), T-4 
(recorded briefing, audio only), and the A-10 (demonstration). A few T-5 
lEWs characterized recorded briefing as "unreliable," and an equal number 
apparently had never operated the feature. However, most T-5 lEWs 
considered recorded briefinq to operate reliably. It was difficult to 
assess the operational reliabiMty of recorded briefing on the T-4 trainer 
and demonstration on the A-lO simulator, since most of the Sis had never 
operated these features. 
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utility and utilization . The briefing featc^es, where available, 
tended to receive lower utility and utilization ratings than did most of 
the other features. The potential training value ratings suggested that 
these features might have some potential usefulness for EW training, but as 
a group, the briefing AIFs generally received lower potential training 
value ratings than did most of the other AIFs, Instead of structured 
briefings, most Sis appeared to prefer informal briefings, which could be 
adapted to the particular needs of individual students and instructors. 

Mission Control AIFs 

Operational status . Those features that were manually operated and 
simple to use (e.g., freeze, reset) appeared to cause few operational 
problems. This was not the case for the other mission control AIFs. 
Manual malfunction insertion, although not included in the survey, was 
apparently preferred by most Sis over auto malfunction insertion. Whereas 
auto malfunction insertion operated reliably on the 3-52 WST, it was either 
never used or operated unreliably on all other devices surveyed. In 
addition, approximately one-third of the B-52 WST Sis characterized 
programmed and manual threat control as "unreliable." Although these 
features appeared to operate more reliably on the other devices, the only 
device for which no reliability problems were noted was the T-4 trainer. 

Utility and utilization . The ratings of total system freeze and reset 
were consistently high. TFie freezing of an ATD (in order to offer 
feedback) and the subsequent resumption of a training mission appeared to 
be a critical capability of all ATDs surveyed. Partial freeze, which was 
available only on the B-52 WST and TAC ATDs, was used less frequently but 
was nevertheless considered to be at least moderately useful for EW 
training. 

Auto malfunction insertion was used with moderate regularity 
(averaging about once a mission) during ATC and SAC missions. In contrast, 
it was considered less important for TAC missions and, consequently, was 
used less often. Indeed, TAC Sis much prefer the discretionary use of 
malfunctions. This may be a more appropriate training strategy in their 
case since the TAC missions are typically more dynamic and less structured 
than those of ATC and SAC. 'Consistent with this strategy, the F-4G lEWs 
generally assigned higher ratings to manual threat control than to 
programmed threat control, whereas the ATC and SAC Sis generally assigned 
similar and extremely high ratings to both features. 

Monitor and Feedback AIFs 

Operational status . The operational status of these features varied greatly 
across the devices surveyed. The A-10 simulator is the only device that includes 
a record/playback capability. Most of the A-10 RTU Sis never even operated the 
feature. However, most of the A-10 CTU Sis did operate the feature, and none 
reported any problems in reliability. Over 70% of the B-52 WST Sis indicated that 
their record/playback capability was unreliable. It should be noted, however, 
that record/playback is not available from the defensive station. During an 
integrated training session, only aircraft position information is available to 
the defensive station during a playback. There is no playback of defensive crew 
actions. These indications appeared to reflect deficiencies in software rather 
than hardware. As noted in a previous section of this report, many B-52 WST Sis 
commented that it wa'; impossible to replicate a particular threat scenario on the 
WST because threats are generated probabilistically. 
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The operational reliability of the monitor and feedback AIFs appeared 
to be greatest for the T-5 and T-4 trainers. There were no "'unreliable" 
indications for any of the features. In contrast, the reliabnity of the 
monitor and feedback AIFs on the B*52 UST appeared to be especially 
lacking. 

Utility and utilization . In general, record /pi ay back and hard copy, 
where available, received the lowest utility and utilization ratings. In 
contrast, procedures and parameters monitoring tended to be aniong the 
highest rated of all features surveyed. This generalization did not apply 
to the A-10 CTU Sis, however, who assigned relatively low frequency-of-use 
and training value ratings to procedures and parameters monitoring. The 
low ratings were due in part to the large number of A-10 CTU Sis who 
indicated "no such capability" for these features, although it Is not clear 
why this was the case. The A-10 CTU Sis also assigned low ratings to 
electronic warfare performance scoring because of problems In reliability. 
Overall, the potential usefulness of the monitor and feedback features was 
relatively high. The means ranged from "moderately useful" to "extremely 
useful." 

Differences Among the ATDs 

The survey revealed many differences among the ATDs in terms of AIF 
utility, utilization, and operational status. Some of these differences 
were noted in the previous section, but several general observations can 
also be made. An overview of the operational status data (Tables 5, 9, 14, 
19, 24) suggests that the T-5 trainer was the most reliable device, 
followed, in order, by the T-4 trainer, F-46 simulator, B-52 VIST, and A-10 
simulator. The devices were similarly ordered with respect to AIF utility 
and utilization. The T-5 and T-4 trainers tended to receive the highest 
ratings, whereas the B-52 WST and the A-10 simulator tended to receive 
lower ratings. Paradoxically, the more sophisticated devices (I.e., those 
devices with the more extensive Instructional support capabilities) 
received the least favorable evaluations. This paradox may not be 
characteristic of ATDs in general, but It leads us to ask whether effective 
EU training might be better achieved through the use of part-task or 
specialized trainers. 

Differences Between the Two Levels of Training 

The pattern of AIF utility and utilization was, for the most part, 
similar across the two levels of training. Some differences were observed 
among the TAC Sis, however. The F-4G RTU Sis used manual threat control 
more often than all other features, and the A-10 RTU Sis used procedures 
monitoring, parameters monitoring, and electronic warfare performance 
scoring significantly more often than did the A-10 CTU Sis. The A-10 RTU 
Sis also rated parameters monitoring to be more useful for training than 
did the A-10 CTU Sis, These differences were consistent with the 
particular characteristics of the TAC RTU missions, which are less 
structured and more closely monitored than were the CTU missions. The 
failure to find RTU-CTU differences within SC probably reflected the fact 
that these missions were more similar to one another than those of TAC. 
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Predicting the Frequency of A IF Use 



The multiple linear regression analyses (see Tables 8, 13, 18, 23, 28) 
indicated that at least 25X, and as much as 47X, of the variability in the 
frequency- of-use ratings could be explained by the remaining variables. 
Training value was clearly the most significant predictor at every ATD site 
surveyed. Indeed, it was the only rignificant predictor at three of the 
five sites. Adequacy of training received added a small but significant 
degree of predictability to the T-4 and B-52 WST regression equations. 

What can be concluded from these facts? Unfortunately, correlational 
findings do not logically imply causality; they merely reflect the presence 
of a relationship between variables. In this case, however, it seems 
reasonable to assume that particular AIFs were used more frequently because 
they were more useful. Indeed, assuming that the training value of an AIF 
did not affect its use is clearly implausible. 

How can the fact be explained that the remaining variables (i.e., ease 
of use, adequacy of training received) did not account for much of the 
variability in frequency of use? This fact suggests that the Sis would not 
avoid using a particular feature, even if it were complicated to use, as 
long as they believed that it would help accomplish mission objectives. 

Training Received by the Simulator Instructors 

The ratings suggested that the Sis considered their training to be 
more adequate than inadequate. The T-4 Sis gave the highest ratings, 
followed, in order, by the T-5, F-4G, B-52 WST, and A-10 Sis. There 
appeared to be little difference between the adequacy of training received 
in the operation and effective use of the features. 

There was considerable variability in the amount and type of training 
received by the Sis. The amount of formal initial training appeared to be 
related to the magnitude of the utility and utilization ratings. For 
example, the T-4 and T-5 Sis received the greatest amount of formal initial 
training, and the T-4 and T-5 ATDs were the most favorably evaluated 
devices. The importance of refresher training was less clear. Thus, the 
T-4 and A-10 Sis received the greatest amount of refresher training, but 
thp ratings of their respective devices were quite different. 

Comparisons Between Phases I. II, and III 

Due to the differences between the questionnaires used in Phase III 
and in Phases I and II, it is difficult to directly compare the ratings. 
For example, the frequency-of-use ratings obtained from the Phase III 
questibhnaire were considerably higher than those obtained from the 
previous questionnaires. Nevertheless, it cannot be concluded that 
electronic warfare Sis use instructional features more often than do other 
Sis, because the appropriate question and possible answers were worded 
differently in each case. Thus, in Phase III the Sis were asked in 
Question 1 "During five typical missions, how often did you use each 
instructional feature?" The possible answers spanned seven categories 
ranging from "never" to "8 or more times a mission." In Phases I and II, 
the Sis were simply asked: "How often have you used each instructional 
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feature?" In this case the possible answers ranging from "never" to "most 
oftefi." The other questions were also worded differently. 

It was possible to make certain valid comparisons, however. For 
example, within all phases of the survey, the relative ratings of 
particular AIFs were fairly consistent across all ATDs. That is, those 
features rated highest (lowest) by one group of Sis also tended to be rated 
highest (lowest) by the other groups. This suggests that the overall 
pattern of AIF use is similar across the MAJ(X)Ms. Another valid 
generalization was that the perceived training value of a feature appeared 
to be the single most consistent determiner of its use. 



V. CONCLUSIONS A«) RECOMMENDATIONS 



At the conclusion of Phases I and II, it was recommended that certain 
AIFs need to be made more reliable and user friendly before their trcining 
effectiveness can be ascertained. It was also recommended that an 
intensive training program be established in order to teach Sis how to use 
AIFs more effectively. These recommendations apply to Phase III as well, 
for it is clear that many Sis have not yet fully explored the existing 
instructional capabilities of their ATDs. 

The principles of effective AIF use still need to be specified. Such 
principles will not be derived from surveys but, rather, from empirical 
investigations. Several reports by R. G. Hughes and his coUeaaues 
(Bailey & Hughes, 1980; Bailey, Hughes, & Jones, 1980; Hughes, 1979; 
Hughes, Hannon, & Jones, 1979; Hughes, Lintern, Wightman, & Brooks, 1981) 
do provide conceptual models for AIF-based simulator training programs and 
present experimental evidence aimed at determining the training value of 
particular features, but much additional work is needed if military ATDs 
are to be more than mere substitute weapon systems. 

It is recommended that future procurement of AIFs be preceded by a 
detailed front end analysis that clearly relates AIF capability to training 
needs. The analysis should consider all known training applications of the 
simulator as well as any major constraints in the operational environment. 
During procurement, AIF specifications should be prepared so as to meet 
user needs and ensure equipment reliability. After operatioral deployment, 
the user should provide adequate instructor/operator training in AIF use. 
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ADVANCED INSTRUCriCNAL FEATURES - EWI SURVEY 



Mane Squadron Date 

FLYING EXFERIQICE: 

Aircraft Total Hours Instructor Hours 



SItVUaOR EXPERIEIKE: 

Simulator Total Hours Instructor Hours 



1. What percent of your initial instruction on simulation training consisted of formal 
classroom instruction and what percent consisted of informal instruction? 

% formal classroom % informal 

2. Have you had refresher training on simulation operatiori? yes no 

(If no, skip next two items.) 

a. How Long has it been since you last had refresher training? weeks 

b. What percent of your refresher training was formal and what percent informal? 

% formal classroom % informal 

BRTEFLY DESCRIBE A •^TYPICMi" TRAINING SESSION ON THIS SMJLfllOR: 



(ZXIERAL CQMENrS AND/OR REOOMME^DATIONS : 



ERIC 



50 

12S 



Read the definitions of each instructional feature carefully, in the 
space next to each feature ^ write the single number corresponding to the 
statement that best describes the operational status of that feature: 



0. The simulator has no such capability. 

1. Capability present but I have never seen it operate. 

2. Capability present but unreliable. 

3. Capability present and reliable. 



Instructor Tutorial - provides the instructor with self-paced 
prograonied instruction in the capabilities and use of the simulator. 

— . Rgggt - permits instructor to "return" the sinailated aircraft to a 
stored set of conditions and parameters. 

Total Systgo Freeze - permits instructor to interrupt and suspend 
simulated flight by freezing all system parameters. 

Partial Freeze - permits instructor to freeze various flight 
parametars or parameter oombiMtions such as altitude, heading, 
position, attitude, flight system, etc« 

Reoorded Brief ing - permits instructor bo provide student with 
information about a structured training session through audio/visual 
madia presentation* 

____ Demcnatration * permits instructor to demonstrate optimal electronic 
war;are procedures by prerecording and subsequently playing back a 
a4iimia»^ engagement* 

____ Record/Playback * permits instructor to record and subsequently 
playback a segment of simulated flight. 

Automated Malfunction Insertion • permits instructor to pre-program a 
sequence of aircraft coa^onent malfunctions and/or emergency 
conditions. 

Hard Copy - provides a record of alphanumeric and/or graphic 
performance data for debriefing purposes* 

Progranned Threat Control • conputer^oontroUed standardized training 

sessions based on pr^progranmed event sequences. 

Manual Threat Control - permits instructor to modify threat scenarios 

luring a training session* 

Procedures Mpnitoring - permits instructor to monitor discrete actions 

performed by the student in accordance with a procedurally defined 
checklist* 

Parameters Monitoring • permits instructor to monitor various 
instnnent readingsT'control settings, aircraft states, or 
navigational profiles* 

Electronic Warfare Performance Scoring - provides a performance metric 

that smnarizes the outcomes of EM engagements. 
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FB-lllA OPERATIONAL FLItiHT TRAINER: MEAN RATINGS 
(AND STANDARD DEVIATIONS) OF AIF UTILITY AND 
UTILIZATION BY RADAR NAVIGATOR INSTRUCTORS 
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Table Bl . FB-lllA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Frequency of AIF Use 



Feature 


Transition 


Operational combined 


PAH 


6.8 
(0.6) 


(0.8) 


6./- 




PRH 


6.8 
(0.6) 


6.5 
(0.9) 


6.6 




PTC** 


3.7 

(2.1) 


5.4 

(1.7) 


4.8 


R 


4.0 
(1.3) 


5.1 
(1.0) 


4.7 


PF 


5.3 
(1.2) 


4.3 
(1.7) 


4.7 


TSF 

1 Sr 


3 8 


A 7 
H • J 

(1.2) 


4.1 


RP 


2.3 
(0.8) 


1.8 

(0.7) 


2.0 




0 


1.6 
(0.8) 


1.5 
(0.6) 


1.6 




NO 


1.5 
(0.7) 


1.3 

(0.7) 


1.4 




Combined 


4.0 


4.1 


T7L 



** £ < .01. 
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Table B2. FB-lllA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Ease of AIF Use 



Feature 


Transition 


Operational 


Combined 


DP 

rr 


0.3 


6.2 


6.2 




(0.9) 


(0.9) 


1 Or 


c n 
O.U 


6.1 


6.1 




(1.1) 


(0.8) 


D 
It 




5.7 


5.4 




(1.6) 


(1.0) 




PTr 


A A 


4.5 


4.5 




(1.3) 


(1.4) 


PAM 




A 1 

4.1 


4.4 




(1.3) 


(1.3) 


PRH 


4.6 


3.6 


4.0 




11./; 


u .o; 


RP 


3.9 


3.4 


3.6 




(1.4) 


(1.1) 


D 


3.1 


3.3 


3.2 




(1.6) 


(1.0) 




HC 


2.0 


3.0 


2.7 




(0.8) 


(1.8) 




Unweighted Means 


4.5 


4.4 


4.5 
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Table B3 , FB-lllA Operational Flight Trainer: Mean 

Ratings (and Standard Deviations] of the Amount 
of Training Received In AIF Use 



Feature 


Transition 


Operational Combined 


PAH 


5.3 

/I ^ \ 

(1.1) 


5.6 
(1.2) 


5.5 




PRH 


5.2 
(1.1) 


5.6 
(1.2) 


5.5 




TSF 


4.1 

/I 1 \ 

(1.1) 


4.4 

(1.4) 


~4.3 


PTC 


3.1 
11. 4) 


4.7 
(1.6) 


4.2 


PF 


4.2 

(1.3) 


4.1 

(1.7) 


4.1 


R 


3.6 
(l.Z) 


4.3 
(1.4) 


4.1 


0 


2.4 
(1.4) 


2.3 
(1.5) 


2.3 




RP 


2.5 
(1.3) 


2.1 
(1.1) 


2.2 




HC 


1.4 

(0.7) 


1.4 

(0.7) 


1.4 




Cond)ined 


3.5 


3.8 


3.7 



60 136 



T able B4 . FB-lllA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Training Value of AIFs 



Feature 


Transition 


Operational 


Combined 


PRM 




C 7 


6.5 




(1.1) 


(0.8) 


PAM 






6.4 




(1.1) 


(0.9) 


PF 




A 1 


6.1 




(0.8) 


(1.0) 


TSF 




R Q 


5.8 




(1.1) 


(1.1) 


PTC 




R R 


5.2 




(1.7) 


(1.4) 


R 


4.6 


5.4 


5.2 




(1.6) 


(1.1) 


0 


3.3 


3.7 


3.5 




(1.9) 


(1.4) 


RP 


3.8 


3.3 


3.5 




(1.5) 


(1.4) 


HC 


2.2 


3.1 


2.8 




(0.4) 


(1.4) 




Unweighted Maans* 


4.7 


5.1 


5.0 



*£ < .05. 
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Table B5. FB-UIA Operational Flight Trainer: 

Mean Ratings (and Standard Deviations) 
of the Potential Training Value of AIFs 



Feature 



Transit ■'•n 



OperationaT 



Combined 



PRM 

PAM 

PF 

TSF 

PTC 

R 

AMI 
0 

RP 
IPX 
HC 
RB 

Confined 



6.6 

(0.5) 

6.5 
(0.5) 

6.5 
(0.7) 

5.9 

(1.0) 

5.5 

(1.5) 

5.3 
(0.9) 

5.5 
(1.7) 

5.5 
(1.5) 

5.2 
(1.5) 

4.6 
(1.5) 

4.5 
(1.9) 

3.9 

(1.8) 



6.9 
(0.3) 

6.7 

(0.6) 

6.0 
(1.2) 

6.0 
(0.9) 

5.7 
(1.3) 

5.7 
(0.9) 

5.2 

(1.2) 

4.7 
(1.6) 

4.4 
(1.7) 

A. 6 
(1.6) 

4.1 
(1.6) 

4.2 
(1.7) 



6.8 
6.6 
6.2 

6.q_ 

5.6 
5.5 
5.3_ 
_5.0 
4.7 
4.6_ 
4.3 
4.1 



5.5 



5.4 



5.4 
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